Monday, May 20, 2019

Starbucks turns to technology to brew up a more personal connection with its customers

Walk into a Starbucks store anywhere in the world and you’ll encounter a similar sight: coffee beans grinding, espresso shots being pulled and customers talking to baristas while their coffee order is hand-crafted.

The process may look like a simple everyday scene, but it is carefully orchestrated to serve Starbucks’ more than 100 million weekly customers. With the help of Microsoft, Starbucks is creating an even more personal, seamless customer experience in its stores by implementing advanced technologies, ranging from cloud computing to blockchain.

“We have a world-class team of technologists engaging in groundbreaking innovation each day. Their inventiveness and intellectual curiosity are matched by their dedication to enabling the Starbucks experience, and this is increasingly critical to how technology has to show up for us,” says Gerri Martin-Flickinger, Starbucks executive vice president and chief technology officer.

“Everything we do in technology is centered around the customer connection in the store, the human connection, one person, one cup, one neighborhood at a time.”

At the Microsoft Build 2019 conference, Microsoft CEO Satya Nadella recently demonstrated how Starbucks delivers its signature customer experience with new technologies.



Making recommendations more relevant with reinforcement learning


Starbucks has been using reinforcement learning technology — a type of machine learning in which a system learns to make decisions in complex, unpredictable environments based upon external feedback — to provide a more personalized experience for customers who use the Starbucks® mobile app.

Within the app, customers receive tailor-made order suggestions generated via a reinforcement learning platform that is built and hosted in Microsoft Azure. Through this technology and the work of Starbucks data scientists, 16 million active Starbucks® Rewards members now receive thoughtful recommendations from the app for food and drinks based on local store inventory, popular selections, weather, time of day, community preferences and previous orders.

“Just like their relationship with a barista, customers receive the same care and personalized recommendations when it comes from our digital platforms,” says Jon Francis, senior vice president, Starbucks Analytics and Market Research.

This personalization means that customers are more likely to get suggestions for items they will enjoy. For example, if a customer consistently orders dairy-free beverages, the platform can infer a non-dairy preference, steer clear of recommending items containing dairy, and suggest dairy-free food and drinks.

In essence, reinforcement learning allows the app to get to know each customer better. And while the recommendations are driven by a machine, the end goal is personal interaction.

“Starbucks is an experience,” says Martin-Flickinger. “And it’s centered around that customer connection in the store, the human connection, one person, one cup, one neighborhood at a time. I think that mission is so critical to how technology has to show up for us.”

Now, Starbucks is looking to expand this technology to the drive-thru experience.

“As an engineering and technology organization, one of the areas we are incredibly excited to be pursuing is using data to continuously improve the experience for our customers and partners,” says Martin-Flickinger. “Using data for personalization is vital to our mobile app, and now we are leveraging data to improve our drive-thru experience.”

Because the technology does not have the individual order histories for drive-thru customers that are available for mobile app customers, it will generate relevant drive-thru recommendations based on store transaction histories and more than 400 other store-level criteria. These recommendations will be offered proactively on a digital menu display from which customers can order. Eventually, customers will be able to explicitly opt in to recommendations that are even more personalized.

Starbucks is currently testing this technology in its Tryer Center innovation hub in Seattle, with plans to roll it out soon. And according to Francis, reinforcement learning will continue to have an important role at Starbucks in many other applications going forward.

“We’re meeting our customers where they are — whether in-store, in their car or on the go through the app — using machine learning and artificial intelligence to understand and anticipate their personal preferences,” he says. “Machine learning also plays a role in how we think about store design, engage with our partners, optimize inventory and create barista schedules. This capability will eventually touch all facets of how we run our business.”

Implementing IoT to deliver a smooth coffee experience


 Each Starbucks store has more than a dozen pieces of equipment, from coffee machines to grinders and blenders, that must be operational around 16 hours a day. A glitch in any of those devices can mean service calls that rack up repair costs. More significantly, equipment problems can potentially interfere with Starbucks’ primary goal of providing a consistently high-quality customer experience.

“Any time we can create additional moments of connection between our partners and customers, we want to explore and activate,” says Natarajan “Venkat” Venkatakrishnan, vice president of global equipment for Starbucks. “Our machines are what allow our partners to create that special beverage, and ensuring they are working properly is critical.”

To reduce disruptions to that experience and securely connect its devices in the cloud, Starbucks is partnering with Microsoft to deploy Azure Sphere, designed to secure the coming wave of connected internet of things (IoT) devices across its store equipment.

The IoT-enabled machines collect more than a dozen data points for every shot of espresso pulled, from the type of beans used to the coffee’s temperature and water quality, generating more than 5 megabytes of data in an eight-hour shift. Microsoft worked with Starbucks to develop an external device called a guardian module to connect the company’s various pieces of equipment to Azure Sphere in order to securely aggregate data and proactively identify problems with the machines.

The solution will also enable Starbucks to send new coffee recipes directly to machines, which it has previously done by manually delivering the recipes to stores via thumb drive multiple times a year. Now the recipes can be delivered securely from the cloud to Azure Sphere-enabled devices at the click of a button.

“Think about the complexity — we have to get to 30,000 stores in nearly 80 markets to update those recipes,” says Jeff Wile, senior vice president of retail and core technology services for Starbucks Technology. “That recipe push is a huge part of the cost savings and the justification for doing this.”

The overarching goal with Azure Sphere, Wile says, is to shift from reactive maintenance to a predictive approach that heads off issues before they happen. Longer term, the company envisions leveraging Azure Sphere for additional uses such as managing inventory and ordering supplies, and will encourage suppliers of its devices to build the solution into future versions of their products.

Using blockchain to share coffee’s journey with customers


Starbucks is also innovating ways to trace the journey that its coffee makes from farm to cup — and to connect the people who drink it with the people who grow it.

The company is developing a feature for its mobile app that shows customers information about where their packaged coffee comes from, from where it was grown and what Starbucks is doing to support farmers in those locations, to where and when it was roasted, tasting notes and more.

For Starbucks, which has long been committed to ethical sourcing, knowing where its coffee comes from is not new. Last year alone, Starbucks worked with more than 380,000 coffee farms. However, digital, real-time traceability will allow customers to know more about their coffee beans. Perhaps even more important and differentiating are the potential benefits for coffee farmers to know where their beans go after they sell them.

Sunday, February 24, 2019

Microsoft at MWC Barcelona: Introducing Microsoft HoloLens 2

This is a tremendously exciting time for Microsoft, our partners, our customers, the computing industry and indeed the world. The virtually limitless computing power and capability of the cloud combined with increasingly intelligent and perceptive edge devices embedded throughout the physical world create experiences we could only imagine a few short years ago.

When intelligent cloud and intelligent edge experiences are infused with mixed reality, we have a framework for achieving amazing things and empowering even more people.

Today represents an important milestone for Microsoft. This moment captures the very best efforts and passion of numerous teams spanning Azure, HoloLens, Dynamics 365 and Microsoft Devices — this truly is a moment where the sum is greater than the parts. From cutting-edge hardware design to mixed reality-infused cloud services, today’s announcements represent the collective work of many teams. And none of this would be possible without our passionate community of customers, partners and developers.

On behalf of everyone on the team, it is my privilege to introduce you to HoloLens 2 and all the announcements we made today to kick off MWC Barcelona.

Introducing HoloLens 2




Since the release of HoloLens in 2016 we have seen mixed reality transform the way work gets done. We have unlocked super-powers for hundreds of thousands of people who go to work every day. From construction sites to factory floors, from operating rooms to classrooms, HoloLens is changing how we work, learn, communicate and get things done.

We are entering a new era of computing, one in which the digital world goes beyond two-dimensional screens and enters the three-dimensional world. This new collaborative computing era will empower us all to achieve more, break boundaries and work together with greater ease and immediacy in 3D.

Today, we are proud to introduce the world to Microsoft HoloLens 2.

Our customers asked us to focus on three key areas to make HoloLens even better. They wanted HoloLens 2 to be even more immersive and more comfortable, and to accelerate the time-to-value.

Immersion is greatly enhanced by advancements across the board, including in the visual display system, making holograms even more vibrant and realistic. We have more than doubled the field of view in HoloLens 2, while maintaining the industry-leading holographic density of 47 pixels per degree of sight. HoloLens 2 contains a new display system that enables us to achieve these significant advances in performance at low power. We have also completely refreshed the way you interact with holograms in HoloLens 2. Taking advantage of our new time-of-flight depth sensor, combined with built-in AI and semantic understanding, HoloLens 2 enables direct manipulation of holograms with the same instinctual interactions you’d use with physical objects in the real world. In addition to the improvements in the display engine and direct manipulation of holograms, HoloLens 2 contains eye-tracking sensors that make interacting with holograms even more natural. You can log in with Windows Hello enterprise-grade authentication through iris recognition, making it easy for multiple people to quickly and securely share the device.

Comfort is enhanced by a more balanced center of gravity, the use of light carbon-fiber material and a new mechanism for donning the device without readjusting. We’ve improved the thermal management with new vapor chamber technology and accounted for the wide physiological variability in the size and shape of human heads by designing HoloLens 2 to comfortably adjust and fit almost anyone. The new dial-in fit system makes it comfortable to wear for hours on end, and you can keep your glasses on because HoloLens 2 adapts to you by sliding right over them. When it’s time to step out of mixed reality, flip the visor up and switch tasks in seconds. Together, these enhancements have more than tripled the measured comfort and ergonomics of the device.

Time-to-value is accelerated by Microsoft mixed reality applications like Dynamics 365 Remote Assist, Dynamics 365 Layout and the new Dynamics 365 Guides applications. In addition to the in-box value, our ecosystem of mixed reality partners provides a broad range of offerings built on HoloLens that deliver value across a range of industries and use cases. This partner ecosystem is being supplemented by a new wave of mixed reality entrepreneurs who are realizing the potential of devices like HoloLens 2 and the Azure services that give them the spatial, speech and vision intelligence needed for mixed reality, plus battle-tested cloud services for storage, security and application insights.

Building on the unique capabilities of the original HoloLens, HoloLens 2 is the ultimate intelligent edge device. And when coupled with existing and new Azure services, HoloLens 2 becomes even more capable, right out of the box.

HoloLens 2 will be available this year at a price of $3,500. Bundles including Dynamics 365 Remote Assist start at $125/month. HoloLens 2 will be initially available in the United States, Japan, China, Germany, Canada, United Kingdom, Ireland, France, Australia and New Zealand.

In addition to HoloLens 2, we were also excited to make the following announcements at MWC Barcelona.

 Azure Kinect Developer Kit (DK)




The Azure Kinect DK is a developer kit that combines our industry-leading AI sensors in a single device. At its core is the time-of-flight depth sensor we developed for HoloLens 2, high-def RGB camera and a 7-microphone circular array that will enable development of advanced computer vision and speech solutions with Azure. It enables solutions that don’t just sense but understand the world — people, places, things around it. A good example of such a solution in the healthcare space is Ocuvera, which is using this technology to prevent patients from falling in hospitals. Every year in the U.S. alone, over 1 million hospital patients fall each year, and 11,000 of those falls are fatal. With Azure Kinect, the environmental precursors to a fall can be determined and a nurse notified to get to patients before they fall. Initially available in the U.S. and China, the Azure Kinect DK is available for preorder today at $399. Visit Azure.com/Kinect for more info.

Dynamics 365 Guides


When we announced Dynamics 365 Remote Assist and Dynamics 365 Layout on October 1, we talked about them as the “first” of our mixed reality applications for HoloLens.

Today, we are proud to announce our latest offering: Microsoft Dynamics 365 Guides.

Dynamics 365 Guides is a new mixed reality app that empowers employees to learn by doing. Guides enhances learning with step-by-step instructions that guide employees to the tools and parts they need and how to use them in real work situations. In addition to the experience of using Guides on HoloLens, a Guides PC app makes it easy to create interactive content, attach photos and videos, import 3D models and customize training to turn institutional knowledge into a repeatable learning tool.

This application will help minimize downtime and increase efficiency for mission-critical equipment and processes and becomes the third Dynamics 365 application that will work on both the previous generation of HoloLens and the new HoloLens 2.


Azure Mixed Reality Services


Today we also announced two new Azure mixed reality services. These services are designed to help every developer and every business build cross-platform, contextual and enterprise-grade mixed reality applications.

  •  Azure Spatial Anchors enables businesses and developers to create mixed reality apps that map, designate and recall precise points of interest that are accessible across HoloLens, iOS and Android devices. These precise points of interest enable a range of scenarios, from shared mixed reality experiences to wayfinding across connected places. We’re already seeing this service help our customers work and learn with greater speed and ease in manufacturing, architecture, medical education and more.
  • Azure Remote Rendering helps people experience 3D without compromise to fuel better, faster decisions. Today, to interact with high-quality 3D models on mobile devices and mixed reality headsets, you often need to “decimate,” or simplify, 3D models to run on target hardware. But in scenarios like design reviews and medical planning, every detail matters, and simplifying assets can result in a loss of important detail that is needed for key decisions. This service will render high-quality 3D content in the cloud and stream it to edge devices, all in real time, with every detail intact.


Azure Spatial Anchors is in public preview as of today. Azure Remote Rendering is now in private preview in advance of its public preview.

Microsoft HoloLens Customization Program


HoloLens is being used in a variety of challenging environments, from construction sites and operating rooms to the International Space Station. HoloLens has passed the basic impact tests from several protective eyewear standards used in North America and Europe. It has been tested and found to conform to the basic impact protection requirements of ANSI Z87.1, CSA Z94.3 and EN 166. With HoloLens 2 we’re introducing the Microsoft HoloLens Customization Program to enable customers and partners to customize HoloLens 2 to fit their environmental needs.

The first to take advantage of the HoloLens Customization Program is our long-standing HoloLens partner Trimble, which last year announced Trimble Connect for HoloLens along with a new hard hat solution that improves the utility of mixed reality for practical field applications. Today it announced the Trimble XR10 with Microsoft HoloLens 2, a new wearable hard hat device that enables workers in safety-controlled environments to access holographic information on the worksite.

Open principles


Finally, as we closed things out, Alex Kipman articulated a set of principles around our open approach with the mixed reality ecosystem.

We believe that for an ecosystem to truly thrive there should be no barriers to innovation or customer choice.

To that end, Alex described how HoloLens embraces the principles of open stores, open browsers and open developer platforms.

To illustrate our dedication to these principles, we announced that our friends at Mozilla are bringing a prototype of the Firefox Reality browser to HoloLens 2, demonstrating our commitment to openness and the immersive web. Alex was also joined by Tim Sweeney, founder and CEO of Epic Games, who announced that Unreal Engine 4 support is coming to HoloLens.

Our experts say about Microsoft Certification Exams