, 23 June 2021

So-called "Just Walk Out" stores and technology take the collection of shopper data to the next level

If you've been following Amazon’s move towards having physical storefronts, you probably have seen the news about a series of different types of retail stores they have created, including bookstores, grocery stores, general merchandise stores, and shops selling prepared food — each of these being along with the fact that they've owned Whole Foods Markets for the past four years. Let’s take a closer look at the way that these Amazon outlets collect customers' money, how they access their data, and some of the privacy implications tied to Amazon's "Just Walk Out" technology.

The category of smart retail checkout technologies is expected to process nearly $400 billion in transactions by 2025, up from $2 billion in 2020, according to analysts from Juniper Research. So clearly, these technologies are here to stay.

There are several ways to pay when you shop at one of these stores. Let’s classify them into two categories: Category A are like other retailers that have checkout clerks to scan your items and ask you for payment, either by credit card, cash or your smartphone. What Amazon has added is their own Go app, to augment apps such as Apple and Google Pay. Category B – what I'll refer to as "walkout stores" — have completely contactless options that don’t involve any checkout clerks.

Amazon Go vs. Amazon One

Category B stores have two payment methods:

The first is to download and use the Amazon Go app. The app works like many point-of-sale systems (such as Toast, Clover and Square, or the variety of supermarket self-checkout systems) that scan a QR code to associate you with the stream of purchases that you get while shopping in the store.

Here's where it gets complicated. The Go app, used in an Amazon Go store, is more than just another contactless payment app. Its major difference from the regular supermarket scanners is that you don’t have to scan anything when you check out. As you walk around the store and collect your items, Amazon is figuring out what you have in your shopping cart and charges you accordingly once you leave. You just walk out the door. How are they watching, you may wonder? There are numerous cameras on the ceiling (think of your average Vegas casino, only more obvious).

CIBO Express cameras are watching you shop. (Image credit: David Strom)

The Go store model isn’t the only place watching you shop. There are numerous other tech solutions, including Trigo (which uses AI-powered computer vision) and Trax (which uses specialty-built shopping carts) that track your purchases.

A second and more recently developed payment method is called Amazon One. This is only currently available at about 25 stores located in Seattle, New York City and Washington, D.C. It operates at Whole Foods and Amazon Go, Four Star and Amazon bookstores. There's a turnstile-like device that initially associates your credit card and mobile phone number with your unique palm biometric signature. It reminds me about the things you see at airport gates to check you into your flight, only there, the machines scan the barcodes on your boarding pass.

After the initial set up, every time you visit a store with the One system, you’ll just scan your palm when you enter (and leave) the store for about a second or so. Amazon One has chosen to scan the palm print because “you can’t determine a person’s identity by looking at an image of their palm,” as is promised in one of their early press releases.

What's the actual shopping experience like?

I asked friends of mine in New York and Seattle to give Amazon One a try. I also had an opportunity to try out a hybrid model that Amazon has licensed to the CIBO Express airport shops in Newark Airport. (Here's a link to a full review of their shopping experience.) The CIBO stores also have "Just Walk Out" technology, but instead of a palm print, they associate a credit card (and your email address) with your shopping cart. Here's an example of what I bought when I was there last week (and yes, airport prices are high):

My CIBO receipt. (Image credit: David Strom)

It took me a few moments to figure out how to navigate the shop and where to key in my email address. Within a few minutes of leaving the store, I got my receipt. I have to admit — it did feel a bit odd simply walking out of the store and into the airport terminal.  

My friend in Seattle went to his local Whole Foods that was using Amazon One. His biggest challenge was finding the kiosk to sign up for the service, which was hidden temporarily by the Amazon staffer who was helping to sign up customers.

An Amazon One enrollment kiosk at Whole Foods. (Image credit: David Strom)

Part of the challenge for Amazon One is that it only accepts a few credit cards, which in my friend's case, fortunately included his Amex card. “It was actually at least as fun as kindergarten, and the whole process took about five minutes, including kibitzing with the staffer,” he later told me of the experience. After selecting his groceries, he scanned his palm at the normal checkout lane. “It was an interesting and easy way to exchange my biometric data for nearly instantaneous payment opportunities.” 

Privacy concerns and suggestions

These new methods come with an undeniable creepiness factor. Some of this is generated from how Amazon is tracking its own warehouse workers (as this recent New York Times investigation reveals), but also because you're also being tracked and your purchases are being recorded to a database somewhere in Amazon’s cloud. But does that mean that you're going to give up the convenience of online shopping? I don’t think many of us pay much attention to the fact that all of our purchases — and even some of our browsing history — is readily available to Amazon so that they can make recommendations on future purchases. But using "Just Walk Out" tech, our purchase history is now being recorded deliberately and in real time, because the tech is literally watching over our shoulders. 

One privacy advocate has already put out this website to track which retailers use or have indicated that they will use facial recognition. It's the work of the privacy group Fight for the Future and its deputy director Evan Greer has said that people should have a right to pay for things without subjecting themselves to surveillance.

But the creepiness factor goes up when you consider that the new Amazon technologies — and to some extent, the Trigo and Trax algorithms, too — can also be used to record what you almost selected to buy but didn’t. This is an important distinction: think of it as what Amazon stores in your online “wish list”, only creepier, since the vendor has access to your own real-time decision making.

I'm not yet sure of what the impact of this data will be or how it can be used or misused. This blog post from 2018 goes further into tinfoil hat territory and says, “Amazon will be able to create extraordinarily accurate buyer profiles, adding our physical habits to previously known digital ones and consumers are going to have more detailed datasets sitting around on Amazon servers.” We’ll see.

Also upping the creepy is the fact that Amazon One is tracking your palms, although this may not be a big deal to all shoppers. As my friend in Seattle put it, “I already use face recognition and fingerprint ID with my Apple mobile devices, so why not give Amazon access to my palm, too?” The difference is that Apple keeps your biometric data on your phone and doesn’t store any data in the cloud, unlike what Amazon is doing with your palm biometrics. To give you some perspective, the FBI’s National Palm Print System already contains more than 29 million palm prints.

One other class of tech that I haven’t yet touched on can be found in many warehouses, which have been transformed into what analysts are calling micro-fulfillment centers. One of the leading vendors in this space is Fabric, which is being tested by several grocery chains and Walmart. Its goal is to reduce selection and delivery times by using robotic systems that roam the warehouse aisles, rather than human drivers.

Biometric data vs. privacy

While it's true that you don’t need an Amazon account to use the One-based palm scanning entry system, if you ever want to change anything (such as use another credit card or your associated phone number), you will have to associate your Amazon One data with some Amazon account. My recommendation: if you're concerned about your privacy, you should create a new Amazon account with a new email identity just for this purpose.

The Amazon Go app seems appealing, but if you're just going to use it as a regular smartphone app for the aforementioned Category A stores (the ones with point-of-sale systems staffed by actual humans), you might as well stick with Apple or Google Pay, since these apps are available in more places. (Also, when I first tried to set up the Amazon Go app, I ran into problems with my payment card data.)

Of course, if you want to take things to the next level, you could simply go back to using cash or shopping elsewhere. Mark Hurst, who consults on usability and product design, told me he no longer shops at Whole Foods since it was purchased by Amazon and has cut down on his overall Amazon usage. He is steering clear of Amazon One, saying “it is not about checkout, it's about Amazon gaining more biometric data on Americans that it can fuse with other data on behavior and location for its surveillance-capitalist business model. Amazon is destroying Americans' privacy for profit.”

Tristan Louis, who has founded numerous internet startups, agrees with Hurst and told me, “The question is not whether I feel comfortable giving biometric data to Amazon but whether that specific biometric data is the line to be crossed. As a consumer, I believe that the line has been crossed already, and whether it is Amazon, Google, or Microsoft storing that data, it's largely irrelevant. Biometric data used to be a store of some value because it could not be reproduced.”

That's clearly not the case anymore. What happens when a hacker can gain access to this data? Hurst asks, “How will you get yourself a new palm?”