Cameras and sensors monitor shopping baskets, purchases and supermarket route. This is the Smart Retail project financed by the Province of Trento and delivered by Spindox Labs. A focus on keeping an extra eye on shoppers..Cameras and sensors monitor shopping baskets, purchases and supermarket route. This is the Smart Retail project financed by the Province of Trento and delivered by Spindox Labs. A focus on keeping an extra eye on shoppers.
The analysis phase is about to end. In Spindox Labs, the Spindox innovation center founded in Trento two years ago, everything is ready for the launch of a new project dedicated to Smart Retail. Called “SpinRetail,” a 24 month research that aims to invent a new generation of supermarket and with a new shopping experience. Smart Retail means monitoring consumer movements and behaviors within large-scale retail enterprises, thanks to a sophisticated system of tags and cameras combined with shopping baskets and shelves.
SpinRetail’s goal is to identify KPIs that can support retailers in the management of shelves, from set-up to procurement, with an extra eye also to the widespread problem of shoplifting. While on one hand the heatmaps information may suggest a more user-friendly supplying of goods, on the other side, the real-time control of shelves can send alerts both when the quantity of goods will start to run low and when a product, instead of ending up in the cart, slips under a theive’s coat.
Spindox Labs researchers will be supported by Bruno Kessler Foundation with two teams E3DA and TeV. The first team gathers IoT and energy efficiency specialists. Technology of Vision team, on the other hand, mainly deals with AI and counts experts in vision technology and cameras use for the recognition of images and objects (Computer Vision, Image Analysis and Visual Pattern Recognition). Their support will be fundamental for the realization of the Smart Retail subsystem which will collect data on:
1) People’s movements: what are the most crowded routes, what are the relax areas, why certain spaces are chosen and for how long.
2) Shopping cart behavior: which products consumers put in the cart, how they grab and choose them, where they move them or where the products go, and if they are not placed in the basket.
To better understand the developments of SpinRetail, we interviewed Cristiano Carlevaro and Danilo Del Fio, respectively Office Manager and Lead Developer of Spindox Labs.
Spindox Labs and Bruno Kessler Foundation, together for SpinRetail
“The Smart Retail project by Spindox Labs, started on 1st October 2017, will use a system of the so-called Internet of Things, using artificial intelligence techniques, in particular image recognition and object detection (objects recognition within an image)”, Cristiano says.
“The main objective of the system” – continues Danilo – “is to map users’movements through non-invasive technologies, adopting known and commonly used devices, such as smartphones.”
Not only the position of the user and his shopping cart inside the store will be detected. Spindox’s Smart Retail system will also use visual tools to monitor the movement of products within the same cart and on shelves. For this purpose, the technology is capable of measuring pressure, so as to record any insertions or withdrawals of the products from the shelves.
All of this will also help to identify and prevent theft in stores. In fact, when a customer takes a product without adding it to his cart, the system will activate a camera dedicated to the suspect case. “From that moment on, I follow you” adds Cristiano. “If the consumer then places the product, the camera will stop keeping an eye on it.” All this will happen as long as the customer has logged in on the app associated with the system.
But then it will be enough not to launch the app to not get caught? No, on the contrary. SpinRetail will launch a preventive alert for all unidentified customers. In short: a Smart Retail in name and in fact.
Technologies and software architecture
There are several technologies involved in the system, each of which has been studied and chosen for a specific functionality. For the management of indoor location, although there are different technologies that allow to reach different levels of accuracy (from WiFi to RFID, to terrestrial magnetism), Danilo explained how the use of antennas and anchors is optimal for reaching a high level of accuracy. This technology allows to use devices already in possession by users – on which a receptive application to the antenna stimuli, must be previously installed – or ultra low power tags that can be mounted, for example, on shopping carts.
During the analysis phase, Cristiano told us: “we understood which technologies to use and which not, what are the advantages and disadvantages of each: WiFi, too imprecise; RFID could be too expensive because we have to label all the objects. UWB is too expensive and very inaccurate technology. Bluetooth therefore seems to us the most suitable technology for tracking. We based ourselves on literature, on what others have already done. And we bought the best product on the market. So we decided to do tracking with bluetooth low energy (BLE) technology. But we cannot know how it will go until we buy everything we need, test it and find that – we hope not – it is not suitable for doing what we want. If so, we repeat and try again”.
The approach used in this type of project is not the classic one. “You cannot choose the Kanban or Agile methodology, as normally happens in consulting projects. When you develop a research project, you don’t known what will be discovered: you start from an environment analysis, then you begin to reiterate. You cannot follow a specific program”.
“To manage the insertion or withdrawal of products in and from the cart, visual interfaces will be implemented in order to recognize the products and manage them compatibly with the carried out operation,” continues Danilo. An association between the cart and the user who will use it will obviously be required. In this regard, Cristiano has anticipated the intention to insert a member card reader on the cart which permits to unlock the basket instead of using the Euro. This would allow the system to immediately recognize the customer and his consumption habits.
The entire system will make use of infrastructure and architectural components that can manage the huge amount of data generated by the parties involved. It will also be necessary to use machine learning algorithms to recognize the products included in the cart. In addition to the various components, which will be the data sources, there will be modules capable of providing different types of feedback to the actors involved. Management and audit feedback, which will allow you to monitor user behavior, manage any critical issues, configure the system or view statistics on products, users or other useful information. Another type of feedback will be provided to the same consumers through their smartphones.
“Faced with the heterogeneity of the services offered by the system, the software architecture will follow the microservice paradigm”, Danilo specifies. “This architecture will allow the various services involved to scale independently from others as needed”.
ROI, automatic shopping, targeted offers in real time: “Hi, Cristiano. Welcome back!”
Another goal of Cristiano and his colleagues is to develop a Smart Retail solution with a sustainable cost-efficiency ratio for retailers. In short, a product that is marketable and convenient for supermarkets. “In the research project there is a part which explains how much our product costs to the store and what return on investment the supermarket would have to buy it. To keep costs down, we will use a fusion of multiple types of sensors by making a single system, called sensor fusion. But for now we are only in the testing phase, we have to understand what will work”.
In fact, when we ask Cristiano where we are, he replies: “we are in an analysis/development phase, we are going to understand what is the software architecture we need (cloud oriented, microservices). We bought the first tracking system, we have a set of shopping carts in the office. We started training for object cameras. We are photographing and acquiring all the products found in the supermarket to train a machine learning library: we are making the dataset of the product images for the ML library”.
“If my supermarket said to me, “Welcome back, Cristiano. I know you like milk very much, how about an exclusive offer on the next liter of partially skimmed milk?” I think I would be very happy to go back to the store and get a discount.” And, like Cristiano, many will be happy when, thanks to SpinRetail, the stores will be able to implement flexible marketing strategies that are increasingly tailored to their consumers. How, for example, to change product prices in real time based on the characteristics of certain customer segments. Or manage offers in order to make the most of the low turnout days.
