Manage episode 182688945 series 98062
EP092 - Artificial Intelligence Deep Dive
"We're in the middle of an obvious disruption right now: machine learning and artificial intelligence. It is a renaissance, it is a golden age. We are solving problems that were in the realm of science fiction for the last several decades." - Jeff Bezos
This episode is a deep dive into all the use-cases for Artificial Intelligence and machine learning in retail.
- Analytics - Google Automated Insights
- Multi-Variable Regression Testing (Correlation)
- Targeting - Best Audience / Next Best $
- Social Listening / Sentiment
- Campaign Attribution / ROI
- Inventory Management/Forecasting
- Merchandise Compliance
- Checkout - such as Amazon Go
- Product Design - such as Stichfix
- Tagging/Unstructured Data
- Price/Promotion Optimization
- Logistic Optimization
- Drone Delivery
- Natural Language Assistants
- Virtual Agents
- Guided Selling
- Visual Search
- Fitment/Return Avoidance
AI Vendors Discussed
Retail Specific Vendors:
- Twiggle - Search
- Sentient.ai - Visual Search/ Personalization/ Recommendations
- Clarifai.com - Visual Search / Video
- Simbe Robotics - “tally” Robot / Shelf Audit / Inventory
- Focal Systems - Computer Vision / Inventory
- Luminoso - Analytics
Don't forget to like our facebook page, and if you enjoyed this episode please write us a review on itunes.
Episode 92 of the Jason & Scot show was recorded on Monday July 10, 2017.
New beta feature - Google Automated Transcription of the show:
Join your hosts Jason "Retailgeek" Goldberg, SVP Commerce & Content at SapientRazorfish, and Scot Wingo, Founder and Executive Chairman of Channel Advisor as they discuss the latest news and trends in the world of e-commerce and digital shopper marketing.Transcript
Jason: [0:25] Welcome to the Jason and Scott show this is episode 92 being recorded on Monday July 10th 2017 I'm your host Jason retailgeek Goldberg and as usual I'm here with your co-host Scot Wingo.
Scot: [0:40] Hey Jason happy Prime day Eve.
Jason: [0:44] Happy Prime day Eve to you Scott.
Scot: [0:47] You were recording here on July 10th Prime deals have launched the Alexa deals exclusives came out days ago I think I don't like 5 days ago and then now here at 9 to watch somebody else it's pretty exciting.
Jason: [1:00] Yeah if you made a bunch of purchases yet.
Scot: [1:03] Hi man I'm kind of just kind of keeping you in Iowa. One of the things I suffer from that I think I have is already have a fair number of Amazon devices so does seem to be the most discounted items and unfortunately already have a pretty full dance card there.
Jason: [1:19] Yep I'm in the same boat it feels like there's a lot of deals but it's slightly tricky to identify the deals that would be personally interesting to you like as as is a problem with Amazon in many other areas Discovery is not their strength.
Scot: [1:34] Yeah did see some pretty nursing with Deco and sipping pretty dramatically off younis I've seen some 30 40 50 per cent off so it's pretty pretty good.
Jason: [1:43] Yeah if you haven't invested in the hardware this is definitely the right time to buy the hardware and other way to extend till I Kindles and stuff too so the lot of interesting things you can do with Kindle tablets, you can Jailbreak come and put other operating systems and stuff on them so I know a lot of people that use prime day as an opportunity to stock up on Hardware.
Scot: [2:02] This is our first show in July we took a little bit of a vacation there how was your fourth of July.
Jason: [2:09] It was great my mom was in town got to spend some time with her grandson and so we had a good time everyone in my family enjoys the 4th of July except MacGyver who definitely does not enjoy the 4th of July or a dog.
Scot: [2:22] I got to get my ThunderShirt.
Jason: [2:23] We've escalated from the Thundershirt to Pharmaceuticals.
Scot: [2:28] Quaaludes.
Jason: [2:33] Yeah I don't think he's getting quite as strong enough jokes to appreciate it but at least it's helping him take the edge off.
Scot: [2:38] And then I guess you're stuck around Chicago then.
Jason: [2:43] We did.
[2:46] Dumb like mine anything else one of the nice things about Chicago being so flat is all the windows in my home face West and so from any room in our house you can actually see like for commercial firework shows in parallel.
[3:01] Yeah I see you have to go nowhere to enjoy the fireworks.
Scot: [3:06] Courtney retail trips to 247 in the gun down to see the Amazon bookstore there in Chicago.
Jason: [3:13] Been to the Amazon bookstore a few times and I think we've talked about my visit there I have not been to any. Super exciting new retail I was in the Bay Area since our last show and I've talked about the beta store before in Seattle that the original beta store was in Palo Alto so I got a chance to visit that and it was.
[3:37] In a frankly pretty similar to the the Seattle one and I did finally get to visit a. Up in San Francisco I'm a little shame that's been been so so long but I finally got to visit a next generation Apple Store.
Scot: [3:51] What was that all about.
Jason: [3:53] Well it's been pretty widely covered in the Press like these are the stores that have like the expert Grove and they have a lot of the organic elements and have a big video wall these are like the the Angela Earnhardt. Next generation Apple stores in you know I think they are improvements. I don't know that they make a big deal of difference in Amazon's business model like seems like they have the same voluminous number of people and employees as. The traditional Apple Store so I'll be curious if they ever came out and said like but the it's a much more expensive store to build I'd be curious if they feel like they like their. Never better return.
Scot: [4:39] Got it cool sounds like you had a good time where you able to see any good movies.
Jason: [4:45] I am no I did not were woefully behind on movie so Wonder Woman's at the top of the list of ones I haven't seen I feel like being a parent of a toddler is very detrimental to my movie-watching.
Scot: [4:58] Absolute you're way behind on your movie-going.
Jason: [5:01] I am I am I'm jealous of you and your ear like premieres like you can take your kids to the premier's.
Scot: [5:09] Yep you have already seen Spider-Man Despicable Me 3 caught up.
Jason: [5:14] Is a lot of talk about how the Spider-Mans are are much worse than the last generation is that your take.
Scot: [5:22] I like I like this one in the Tobey Maguire I didn't like the one in the middle so I guess I'm counter Spidey.
Jason: [5:30] Got you yeah so there's some there's some like critical videos that have gained traction on the internet that compare the Tobey Maguire ones to these current ones and they there they come down pretty hard on the current ones.
Scot: [5:49] Cool solicitors we've been doing a lot of interviews lately and it's time to mix it up and we're going bring back one of our most popular segments.
[6:13] Deep dive this week we're going to do a deep dive into all things artificial intelligence and how it may impact Commerce.
[6:23] This year's annual letter to shareholders Jeff Bezos talked a lot about Ai and machine learning so here's a little segment from that. These pictures are not that hard to spot they get talked and written about a lot but they can be a strangely hard for large organisations to embrace we're in the middle of an obvious one right now machine learning in artificial intelligence. It's a Renaissance a golden age Bezos said we're solving problems with machine learning in AI that were in the realm of Science Fiction for the last several decades. So I also remember when Bezos kind of dropped in one of those interviews earlier in the year that they had a thousand people working on machine learning so. Jason this one is squarely in your real house so I'm going to kind of take back burner here in simply interview you for for the audience so once you kick it off and give us your definition. Of a i and she running a lot of people using these all over the place so it's want to hear your your kind of foundational of you of of how we should think about these things.
Jason: [7:18] I do think the definitions are all over the place in that that creates a lot of confusion there sort of the, a technical Definition of artificial intelligence which is not what anyone in our industry means when they talk about artificial intelligence cuz they like. Real artificial intelligences was called artificial general intelligence or a GI that's the whole notion of a, computer being able to do all the tasks that a human can and being like you know technology being indistinguishable from a human being and so, nothing that we're talking about is anything approaching that and there's certainly like that technology is not in the near Horizon for us that's. You know at least 10 plus years out and their lot of people that. Smarter people to me that argue about if and when it'll ever happen and if it did you know you could have any get to that. That Singularity that Ray Kurzweil likes to talk about.
[8:14] So most of the time in our industry one they're talking about with the Ruby talking about is applied AI or what the scientist sometimes call Nero AI or weak Ai and what they mean by that is. The Machine's ability to do one specific thing as well as a human being can. And so you know a classic example of of Nero AI is Siri. Being able to do a very specific set of tasks like a human can in this. Highlights the real problem with the definition of a eye is unless you also Define the set of tasks you're talking about.
[8:55] You can't really understand what someone means when they're when they're talking about applied AR right so if.
[9:03] I said like in the 1970s that hey we just invented a computer program that can play chess right like the. Back then the the the Nero task was the ability to follow the rules of chest it wasn't necessarily good at chess and couldn't beat a good chess player but just being able to play chess was a very classic definition of AI in the 1970s. Today for for any of us to really think of Chess as AI you have to be talking about a chess program that can beat a Grandmaster. Right and so the the task that you're talking about change dramatically from just playing chess to playing Chessa to Grandmaster level. And so it it's kind of interesting the AI is always shifting when when you know recommendation engines for e-commerce first came out, that was state-of-the-art AI you know when folks like Netflix and Amazon first launched those features, that that was the The Pinnacle of AI you know today you know you've got a dozen vendors you can pick to plug into your website to do basic product recommendations and most of us don't think of those as. A a current example of a I-44 example so, the definitions are constantly shifting and then we have this problem of their these three terms that get used kind of interchangeably in our industry there's, artificial intelligence which is what we've been saying so far there's a related discipline called machine learning that gets used interchangeably with artificial intelligence a lot and then there is this third term.
[10:35] Cognitive Computing and the there are specific definitions of each of those but when you know in the in the world of, e-commerce and vendors they're all using this using them and using them interchangeably I'm in so it makes it really hard to, know what folks are are even talking about.
Scot: [10:55] So that's helpful I think the thing that the listeners price struggle with is how much is reality and how much is hype so for example when we were at shoptalk if just a couple months ago really, every vendor there for so there's this explosion of new vendors so if we had a fair number of vendors are in history and now there's no. Really a doubling or tripling and it seems like every one of the vendors is a redo of an existing vendor but with a machine like machine learning a I kind of an angle so now there's on site search. Adword bidding machines product recommendations upsell engines email optimizations that.
[11:36] Brazilians of these kinds of things if I'm a retailer. Should part of my 2017 strategy be to just go and figure out all the vendors I have today and find a machine learning version of them and if that's not the answer then where. Where can someone have the biggest impact for for listeners that are out there with us technology.
Jason: [11:55] That's a great question Scot we should do a podcast about that.
Scot: [11:59] We're right in the middle of the chest.
Jason: [12:00] Oh geez alright well I'm going to start while I come up with an answer but, in in all seriousness your hypothetical is I would send it as exactly what you shouldn't do you know there's no reason to just go look for versions of all your turn to experiences, never provided by a vendor that's bolted one of the AI words onto their service because that, that word doesn't make that service any better or worse than it was before and totally agree with you you know vendors are both in these things on right or left like we. You know there's some folks that I be in that take it really seriously but it's fun to poke fun at them, they have this technology or they would call cognitive Computing technology that they branded Watson and some days it feels like they've just added Watson to the front of every product that IBM sells.
[12:51] And so you know the is that a better version than the last version because it has the word Watson in front of it but well. Not necessarily should you pay more money for it because it has the word Watson in front of it like I certainly not. I was looking at the vendor list from irce there's 22 vendors that have bolted a Ion 2. You know their existing product and I'm getting these like calls everyday from vendors saying hey I know you weren't interested on probably before but we pivoted and we're now in a I you know so and so and we would love some of your time to talk about how we should take. Take our product to all your clients and. You know you sort of implied in the question that's a bad strategy nothing's going to be better by just buying an AI version of it. Going back to our friend and number one listened or Jeff Bezos. She talked about machine learning as a sort of a horizontal layer right like so it's not a in point it's a it's a technology that enables, new kinds of experiences and he has this pretty simple definite definition that I like to use he says, like over the past decades computers have broadly automated tasks the programmers could describe with clear rules and Ayala grissom's in what modern machine learning does, is allow us to do the same for tasks where describing the rules is much harder right so, playing chess is a relatively defined set of rules and you could write a computer program that follow those rules but what machine learning let you do is.
[14:30] Make a program that can play chess really well even though the programmer themselves might not be able to write a set of best practices for actually playing chess, and so what what we're really looking for our specific use cases in Commerce, that are made possible or made dramatically better by adding this horizontal layer by adding this ability to, to do fuzzy stuff that was hard to write rules for in the past, and so what I would say rather than looking for labels like you ought to be thinking about specific use cases, that are made much better or an able for the first time by underlying Technologies and decide whether any of those use cases are particular helpful for you.
Scot: [15:20] So that's helpful what what are some. Where some examples of where retailers can use this technology in and maybe give folks a little bit of framework for helping him think about this so that they can kind of formulate a plan and figure out how to start sampling some of these things.
Jason: [15:36] So so what's do exactly that was jump into some specifics and I I like to, kind of divide the experiences into three buckets the first bucket I called the insides generation bucket and that's all of the sort of, analytics data processing type things you can do and I'll go into some examples in just a second the second buzz bucket is what I've called business acceleration it's, saving time or money or reducing complexity from from various business processes. In the third bucket is customer engagement it's it's new customer experiences that you couldn't do before the customers appreciate and make you a better Merchant ER or a better solution for those customers. So let's they're talking about some of the the specific Commerce use cases that might fit in each one of those buck.
[16:32] So the first one I like to talk about in the insights bucket is. Basic web analytics so we've had web analytics for a long time and you know they don't come with key and reports and dashboards and you can make your own custom reports but all of the traditional Analytics. Require you knowing the smart question to ask and then the the analytics engine being able to show you go find the answer to that question you asked.
[17:03] And so you again you could you could put into find rules for what was in that dashboard and what wasn't. What machine learning let you do two analytics is find insights that you weren't smart enough to ask the question for. And so this is already being built into a lot of the traditional analytics product so there is now a beta feature in Google Analytics. Call Google automated insights and essentially instead of you having to define a segment and ask a smart question like. How do mobile users convert versus desktop users or how do first-time visitors convert versus repeat visitors or things like that.
[17:43] Google will use machine learning to evaluate all your data, and suggest segments that that are particularly interesting or highlight some unique opportunities for you so it's. The the analytics engine becoming smart enough to ask the smart questions that we aren't smart enough to ask.
[18:06] For the first time and that's an example to me if something is pretty exciting in the machine learning space that makes Commerce operators much better.
[18:17] So another one that you and I were talking about earlier is this notion of discovering correlations outside of web analytics right so there's there's a lot of. Behavior is in Commerce that that have have.
[18:35] Correlations or there's urban legends that that supposably things correlate that might affect how you run your business so I sort of the the, the famous example in e-commerce is weather and you know that type of product you should offer when it's raining versus Sonny and of course all retailers complain about whatever the weather is in playing that that was the reason that their sales were off. And so it's interesting to know what the correlation the real correlation between weather and sales are the famous not obvious correlation that turns out to be Urban myth is, the beer sales correlate very closely to diaper sales. And you go will guys would have those two have in common and it's it's in theory it was that the the dad got sent to the store to get get a new box of diapers and he also of course grabbed a six pack of beer.
[19:27] And you mentioned you were using some interesting correlation tools at spiffy.
Scot: [19:34] Yes yes sir. My latest company does On Demand Car Wash and detailing and you note small companies still getting off the ground essentially and. So one of our folks was playing around with the Amazon machine learning and the, play the story really is that some of the stuff feels like you have to be a multibillion-dollar company to play with it but we found the Amazon stuff is really approachable we'll put on Lincoln the show notes to took on this model that we used in essentially what you do is you can upload a. Transactional database withing about a really long spreadsheet. Spreadsheet with bunch of transactional data on every row you can put in there what you know about that transaction so obvious things like they OV the skew that kind of stuff in our world of car washing we know the vehicle. We know the location the zip code and some those kinds of things so you know what it's spit out with those really interesting and we also know the weather so. We were just doing this to really kind of.
[20:32] Play around with the weather part of it but it was interesting as it said your inversely correlated to the weather which is the first inside it offered which was too obvious when we were looking for so when it's raining no one wants their car washed, but then the next thing it did and it said your model customer drives an American SUV probably Yukon and.
[20:51] These are the top three zip codes that are correlated to your sales in Sunny warm weather wow those are things we had never even really. Kind of thought that you could figure out but it it is what it does you can come look at that data. And sniff out these correlations that that human just can't process so in all that is done to a pretty simple you I or you can upload a spreadsheet so, why the stuff feels like it's pretty science-fiction E when you hear about it but that was an example that I wanted to share with listeners where we were able to get some pretty interesting insights just by by using a web-based interface Steven API this with Amazon web. Web stuff.
Jason: [21:31] Very cold and so that's that's an actual business user versus a data scientist in that case.
Scot: [21:38] Absolutely.
Jason: [21:38] Awesome yeah so those are those are great examples other common ones that we run into an in Commerce or around like targeting and best audiences so you know again, we have a lot of data about all the people that have bought from you in the past who you know what are the look-alikes that you should be, see you know buying from Facebook or other ad sources that are potentially most valuable to you you know in other all the marketing activities, that you could be doing for your business which one is going to give you the the best return for the next dollar of marketing spend you have so you know we're seeing these, these machine learning based analytics tools, get really good at defining Target audiences and helping figure out next best dollar sort of related to that are, the ability to do attribution and Roy models so you know, traditionally in in e-commerce we all use this model called the last click attribution which is whatever the last thing that guy did before they bought something, that's the activity that got 100% of the credit for the sale. That's kind of the default model in most of the analytics tools still and too many people use it and it's completely wrong headed. You know that sort of like saying like what's the most valuable thing in my store will it's the cash register cuz everyone uses the cash register read before they buy something.
[23:09] The so there are all these other attribution models that give partial credit fractional credit to all the different marketing activities that led up to a purchase in the problem has always been. Will which model you know is most accurate for my business and you had to pick them out all, and you really didn't know if you would pick the right model or not so now with machine learning, the program kind of analyze your data and picks the best attribution model for you and so you know for the first time to your point business users. Using kind of web-based analytics tools can start getting these really sophisticated Roi calculations and customer lifetime value calculations. Without having to be a data scientist that could smartly pick the right attribution model. And then I guess the other area of inside generation that's getting a lot of traction right now. Is this whole notion of sentiment analysis or or more specifically for Commerce will call it social listening right and so that's this. This notion that man you have this fire hose of data of people talking about you on Twitter and Facebook and we chat and.
[24:19] You know should I what should I be doing to enhance my reputation are people talking favorably about me or they speaking negatively about me which tweet should I flag for for customer service follow up. In the old world where you just had to have an army of people read all these things to make decisions on all of it it from most companies. The volume with such that it just didn't scale and didn't make sense but now with machine learning you can actually, process the entire fire hose or social media and do a pretty good job of categorizing all of the the dialogue about your brand or product or business into actionable buckets that tell you, you know. Weather weather audiences are looking at you favorably or negatively with it they like your new products are don't like your new products and more specifically what what specific, comments and social media you should be taking action on a responding to to try to improve your your reputation and customer service.
[25:22] Assume you're doing all that at spiffy Scot.
Scot: [25:25] We're just playing around with some correlations at this point.
Jason: [25:28] Nice I wasn't.
Scot: [25:30] I think some of the cinnamon stuff some of the Wall Street guys are like reading the Twitter firehose to try to get cinnamints on stocks and things it's I'm not sure that use case but it is pretty nursing.
Jason: [25:39] Yeah well I didn't you know once or the interesting one is the.
[25:44] Retailers are starting to report less and less data to the analyst which I know irritates the analyst to no end and so they're looking for all sorts of new tools too sore to get a read, like what weather Retailer's quarterly financial performance will be in any of those are these machine learning tools in some cases it's. Taking pictures of parking lots in malls with drones and using those two to evaluate like whether traffic is up or down in the mall and all sorts of interesting things like that.
Scot: [26:17] Grateful so that's Insight generation than the second bucket you talked about was business acceleration what are some examples that you seen there.
Jason: [26:24] Yeah what's up the classic one that's that's probably the highest Roi today that you see use the most by slightly more sophisticated operators is, the whole machine learning for inventory management in forecasting so you know kind of taking the the, buying a responsibility like out of the hands of the merchant Prince and and you know having them just guess how many of a garment you should make. Or how many you send to each store and instead using the data to sort of accurately tell you, what your inventory level should be in an even more accurately forecast your sales.
[27:08] So there's a whole host of retail and e-commerce tools that are focused on using machine learning for inventory management and forecasting. One that I like that is not quite here yet there's some great demos and a number of retailers are testing at Target I know his testing it. Is what we call merchandising compliance. So if you think about a brick-and-mortar store a lot of the displays in that store are paid for by a brand so, Procter & Gamble might buy an end cap for tide and so the tide is supposed to not just be on the Shelf but beyond the end of the shelf and it supposed to get some special signage and Procter & Gamble probably paid a lot of money. For that in Cap Toe to Walmart or Target or whomever and so in the old days when you when you pay that money, you would then hire a bunch of college students or soccer moms, to go visit every store and take a picture of it and send these report cards back to Procter & Gamble to say whether every individual Target store, complied with that merchandising program or not because you're paying a lot of money for it and what you find is in a significant number of stores, they didn't put they didn't execute the in cap are they didn't put the signage out there used to be a stab that like half of the custom, printed signage that gets sent to the stores the merchandising the temporary point-of-purchase displays never got put out on the shelf and so all the brands had to spend a fortune sending these armies of people out up to the stores.
[28:46] To measure compliance and what we're seeing now is, you can have a Roomba like some kind of robot that roams the the floors of the store with cameras and takes pictures and uses computer vision to match those those pictures against us, the planet that the planograms that the intended store layouts and you can report on, you know which stores did or didn't comply with those displays and you can take corrective action more quickly and you can save all that money of the Brand's having to spend people out to the stores to measure it. And then you can even use those pictures to tell you when, for example all of the particular SQ of tide is out of stock and not on the Shelf cuz it might be in the in the back room and not on the shelf and obviously in that out of stock situation, you're not selling any tide so so using computer vision for merchandise and compliance, you know is it's still early days but there's a ton of money and friction to be saved by doing that. And one of the expenses will talk about next Amazon go, is it a that's sort of one of the underlined capabilities of Amazon go then Amazon go extends that that. Capability by also letting you check out right so the Amazon go store that we've talked about, uses cameras to take pictures of the shells and know what products are on the Shelf but it's also using cameras to follow the Shoppers and know what Shoppers are holding which products so that it can charge them for those products when they walk out of the store.
[30:22] So I would characterize Amazon go as a you know a potential future use case of artificial intelligence for retail.
Scot: [30:32] Regal and then what am I favorite examples is the Stitch fix goddess they talk a lot about private label and one of the reasons they came up with private label was they would they would send all these products people. Would buy them but they would say I liked. The strap on this the design of that in the near able to synthesize all that feedback and essentially the machine morning would say you need to produce this garment. Bob for this audience of people tussle bit more about that.
Jason: [31:03] Yeah I think that's that's a great use case is serve using machine learning for product selection and product design right soap, going back to the kind of old Merchant Prince model you know the Mickey drexler's of the world would decide what what you note, the design of the shirt is that was in gap or J.Crew and you know, it was as much art as it was science and you don't very often you know they would make good good selections and then sell a lot and then make a lot of money but occasionally they would design something the market didn't want and they. Have a ton of it in stock and lose a fortune so what folks like Stitch Fates are doing a saying hey let's not have Merchants what use the data to tell us what products to, to offer to our customers and eventually not just what products to buy and offered our customers but she's the data to decide what, products to design for our customers and offered them and sew and Stitch fits particular case they have like 60 attributes for every garment so, things you wouldn't think of but like how many inches is the top button from the collar of what's the ratio of the waist to the chest in the in the size 6 what you know what what kind of cuffs does it have what kind of treats does it have their defining each garment at a much more granular level of attributes, and then they're using machine learnings to say what are the combination of those attributes in a woman's blouse that sell the best to which of our customers.
[32:35] And so you know originally that use that to the side, Which Wich prod third-party products to carry but more and more of their scent they're using those attributes to Define what products they should manufacture themselves and offer to their customers so it's really replacing the merchant, with with the data and then so instead of having a buyer or Merchant you you have a analyst.
Scot: [32:59] Yeah and then the is a good time to kind of Jack the. The thing that gets pretty nursing your out this is when you think about business models you know one of the favorite business models the last 10 years is Network effects so the classic example is on Marketplace like an eBay where. I order more modern would be maybe an Uber where you have supply and demand and more Supply brings more to me into this flywheel effect happens.
[33:22] So more drivers brings more writers writers brings more drivers or more sellers bring more buyers Etc.
[33:30] I think I think when you start to think alot about this machine learning and II and you use that Stitch fix example that the reason to be able to do that is because they have all this great product data.
[33:40] Data becomes almost the Next Generation Network effect so it's almost like this date and network effect where the more data a company can get about consumer behaviour preferences and those kinds of things they're going to have this Edge that no one else has and.
[33:55] Yeah I know they're kind of called action I. Usually talk about with retailers are especially Brands is this what you need that connection with the customer because imagine your brand is not signed Direct. All that data is out of your hands right now and you're there will be a day when you will be at a severe strategic disadvantage for developing products I think. Trace how you feel about this if you don't have that direct connection to Consumers and you know also not only as the data. Important baby start flexing your muscles around these things are really understanding how to apply so he's techniques to that data.
Jason: [34:28] Absolutely and in so I think you're exactly right with most of the current state-of-the-art machine learning models the big competitive Advantage is, having the data set to train the model and so the more customer interactions you have the more data you have the better model you'll be and the better you're able you be able to serve more customers into your point, you the better your flywheel will be right. An end so that's that's true for a lot of these these different cases and specifically with the manufacturer versus retailer it's. Once this data becomes key you start thinking about whoever owns that relationship with a customer has, access to a way more valuable asset so like one of the examples I always like to use as the tire industry and you think about the the tire manufacturers of the world, for the most part they have no idea what kind of vehicle in what ZIP codes, there their tires getting installed on and they don't know how the tire the the customers use those tires and have no idea for example how long those tires, last on specific vehicles in specific geographies, I'm in so they know they know very few Out reviews about their Tire once it leaves the factory but a good retail that installs those tires on the car can start collecting all these extra attributes, how many miles are on the car what kind of car is it that how frequently do they change their brake pads what ZIP code do they does the car live in and all the sorts of things and that retailer can start using those attributes of that data.
[36:03] To start doing things like much more accurately predicting which Tire will work best for which customer on which vehicle in which geography, and said they're all sorts of interesting things that come into play there and so if you are that that, product manufacturer tire manufacturer whatever like one of your big strategic challenges right now is to figure out how, to start developing that relationship direct with a customer so you can be capturing that data.
[36:36] In the business acceleration and I want to talk about that a little bit more and some of the customer engagement things but there are a couple other business acceleration ones that we should probably just touch on one that's getting used a lot right now is. The idea of tagging or evaluating text, so tons of brands have a lot of texts about their products that they someone typed into a super old database that they used to print the packaging that goes in the store, but the Torah conversation about attributes earlier that wasn't structured data like someone wasn't smart enough to say, we should have a field for whether all these snacks are Kosher or not and we should have the field to say whether all these next are gluten-free or not right like kosher and gluten-free might have just appeared in a, text description somewhere in so there tons of product manufacturers that have, piles of this unstructured data that isn't very useful for machine learning it isn't very useful for search and filtering and all these use cases that are super common in e-commerce and so what you know you either have to, pay a bunch of copywriters to read all your unstructured text and cut and paste it into fields, or you can start using these machine learning models to automatically tag your data and turn unstructured data into valuable attributes. And one of those common when is pictures right so you imagine that you're in a product category that's heavily uploaded to Pinterest or Instagram.
[38:10] You don't know very much about those pictures which which you of yours is in that picture is it being portrayed with a man or woman, is it being for traded a beach or a ski chalet and all these different things that would be interesting to help you decide when to use that image the. Machine learning can tag all of those images and make them much more valuable in in all of your Commerce experiences.
[38:37] So we're trying to see that a lot a common one that's being used right now is almost all of the latest fraud engines. Are using machine learning so this is a classic example where. You know fraud used to be a set of static rules so you would write rules if people try to shop our side in the US from Nigeria we won't let them shop and if they. Try to ship the product to a hotel we won't let them by that. And with machine learning we can be much smarter about what attributes. Trigger a secondary screen for fraud and what that does is it gives you weigh less false positives. So you're able to sell a lot more Goods to a lot more people and not offend them by by treating them like their prospective criminal when they've done nothing wrong, and said that the fraud models are both getting much better at catching fraud but equally important they're getting far fewer false positives as a result of using, this machine learning instead of a set of hard-and-fast rules one of the business accelerations that that Amazon has particularly made famous. Is the whole field of price optimization. And so you know I obviously you don't we talk a lot on the show about Amazon changing 2.5 million prices a day and they're there. Their approach is much more sophisticated than just being the lowest price on everything right like they're there a strategic low price provider and you know more and more of that, it's not possible to to just write a set of rules about what your pricing for every product out of be in so you're starting to see retailers.
[40:14] Turn over the keys to their pricing models to these sophisticated machine learning systems that optimize price and optimize promotions and offers for individual customers and unity earlier Network effect point, those models are most powerful when you're you know at the high end of the volume and you have a ton of transactions and a ton of skews to apply those models against.
Scot: [40:41] Yan is a reminder we had a guest Andrea who was on and relay and she was talking about how. A lot of times even a vendor's negotiating with a robot on the other side and Y episode there's not only are they optimizing the price the consumer sees but there no that's feeding into some engine that then kind of coming back to the vendor and saying you need to price the product at this.
Jason: [41:00] Absolutely and so you know I think Amazon is kind of the gold standard in in Commerce for that and see you're seeing a lot of other like when you, to your point when you had to have a thousand data scientist to write your own pricing on rhythm, you know that that was a huge advantage to the people the top of the echo system like Amazon but. Today you know it is easier to buy an off-the-shelf model, that you just have to have enough data to feed so there's there's vendors out there like Boomerang which are every bit as sophisticated as Amazon's pricing engine but you know it's available too much smaller operators. You know as long as there they have enough data to put into the model and so that super interesting. At the moment the big challenge you have is how do brick-and-mortar retailers do that sort of real-time price optimization like it's pretty easy to change the price.
[41:57] You know from second to second on Amazon it's much harder when there's a paper price tag next to that product. And it's on the Shelf in the store so that's that's an interesting organic when we're going to continue to see play out. Another one that I am azaan is particularly great at is this whole notion of logistics optimization. So once you're bigger than a single Warehouse you start getting you know all these issues about what's the optimum Mount of inventory have in each warehouse and where should you put all that product where is going to be most efficient to get to the most of your customers. And if you're wrong about that that whole supply chain planning you can cuss yourself a fortune moving products around or shipping products inefficiently to customers. And so using machine learning to optimize how many excused and which fuse go into each Warehouse. Is super important in you know when your Amazon and you have what do they have now Scot 112 fulfillment center something like that.
Scot: [42:59] Yep thereabouts.
Jason: [43:01] Yeah that that becomes a. A critical challenge an Amazon spray the only one that has the problem at that scale and they're also probably the only ones that have the solution at that scale. And then I guess the last business acceleration one that you know I don't think we're going to see immediately but gets talked about a lot is. Like obviously all the technology to get that drone to your house to deliver the goods. Is a great example of of something you can only do with artificial intelligence so if we ever see drone delivery be economical for certain customers like that you know that that will be exclusively enabled by. Buy artificial intelligence and machine learning and I would remind listeners whenever I say drone people always imagine these super expensive flying things, we are also starting to see a lot of wheelbase drones and so there's an interesting Pilots going on in in San Francisco and then, Maryland right now with with the drones that are sort of autonomous vehicles that drive on sidewalks and deliver things like pizza and stuff.
[44:10] So I'm excited about renting a house in one of those markets and get a drone pizza delivery.
Scot: [44:16] Call Sears a lot in the business acceleration in the country cap that sounds more like.
[44:23] Cost savings um I guess there's some that impacts the customer experience but the next bucket is where you probably would customers are going to feel it the most which is what you're calling customer engagement.
Jason: [44:34] Exactly and this is the stuff that that tends to be the most sexy it's the most visible to customers and, you know there a lot of things in this category that have their own buzz and then their own own spot in the hype cycle at the moment so one that we talked about a lot or natural language assistance and so that's, you know Siri Cortana Echo, Google home all of those sorts of things and you know if you if you think about them they're actually an amalgamation of multiple a Technologies right like so there's this this notion of being able to convert speech, into data and so their natural language processing and then there's the notion of being able to to. Act on those the sentences, and give proper responses and so that's the notion of virtual assistants right and so you you have a lot of these things that are like you speak to like Siri, you have a lot of virtual agents that you type to a chat box on Facebook and things like that. And you know there's an explosion between those two categories of The Voice assistance and the virtual assistants in in e-commerce at the moment.
[45:53] If you tried any of the the virtual agents Jets Scot you think any of them are ready for primetime.
Scot: [45:59] Now the ones I've tried pretty cheesy and there if you stay with them they're pretty unsatisfying they can't answer most your questions until they kick over to a human 20 minutes better.
[46:14] I'm not believing this are quite there yet.
Jason: [46:16] No and so it is funny because what what we're seeing is, customers definitely want customer service via chat and Via messenger and so it's, a mistake to say oh my gosh the chatbots are kind of not ready for Primetime and so it just hire more phone reps and do everything via phone cuz we're seeing strong indications that customers are less. Tolerant to sit on a hold line and do something asynchronous like like a talk to someone on the phone, but it's same time you're right like the virtual agents really aren't cutting it yet at the moment and so where The Sweet Spot is are our live humans at the other end of those, does chat and SMS strings and I guess the best virtual agents I've seen our kind of maybe just one layer deep and they they.
[47:06] The answer some of the the highest velocity questions and they sort of act as a filter to make those those live agents more efficient by not having them have to answer the same question over and over again.
Scot: [47:18] Yeah and they're smart enough to know when to get out of the way that kind of say hey did you are you looking to track a package oh I'm sorry let me write you to a human.
Jason: [47:27] Exactly right and well that.
Scot: [47:28] Once again a doing there like you know big kind of walk you through of of Sky knowledge-based relentlessly.
Jason: [47:35] The best ones are seamless right and and unfortunately like too many of them you know keep fighting to try to keep you in the virtual realm and you know it some point you you stop asking an honest questions and you're just trying to figure out how to. How to bypass it. So another sort of adjacent thing that we're starting to see more of in customer engagement is this whole notion of Discovery and guided selling and so one of the, the ones that got the most bus here is, North Face uses the Watson implementation to have sort of a guided selling tool for jackets 800-Flowers has a guided selling experience for for gift giving, and, you know I'm a little bit you know I have similar feelings to the guy that selling tools at the moment that you you had to the virtual agents I think the idea of them is very interesting and I I certainly agree. We need to get way better at Discovery and helping people find new products but a lot of the guided selling tools I've seen at the moment just feel to linear and scripted and I'm not sure. The there there yet recommending products a heck of a lot better than then you don't sort of us structured set of rules used to last year.
[48:58] So that the next one in customer engagement is one and I may have even have to go back this may have been one of my predictions so I'm going to type it again in the hopes that it helps my my annual prediction come true. One of the cool Technologies in artificial intelligence is computer vision and being able to. To process images and more more often process video to get insights out of out of that image data. And so one of the the most common use cases for that is tagging images that we talked about him business acceleration but the other way more sexy one is visual search. So that's Amazon Firefly being able to take a picture of a product. And then order it or you know even cooler use case is. Via an app like camfind being able to take a picture of the woman at the table next to you with the cool handbag and find that handbag for sale or those shoes and that's kind of the whole notion of this see it by it kind of experience.
Scot: [50:02] Yeah isn't a lot of people say Pinterest is one of the better ones out there do you know what they're using under the hood for that is it is it some machine learning kind of.
Jason: [50:12] Exactly and they rolled it out that, Pinterest to Lynn's they ruled that out relatively recent like so it's probably only about three months old at this point if memory serves, and that's a great example it's not products Pacific yet so it's, it helps you find similar images to the image or looking at there are some more sort of Commerce e ones I mentioned camfind is one company will talk about a couple other visual search companies at the end of the podcast that the, that the whole field of visual search I would just tell people is getting phenomenally better and so for years we talked about natural language getting. Twice as good every year and last year the natural language interfaces essentially surpassed human comprehension so the, the computers can now more accurately understand spoken words than an average human being and that's you not forgetting the fact that the computers can also understand. People in a bunch of other languages in the same sort of evolution is happening in visual search there's a an academic contest for visual search engines that the several of the universities including Stanford put on every year and the winning visual search engine, is twice as good every year as the year before and so the quality of visual search is doubling every year, so if you look at some of the best use cases right now they're already pretty impressive and you go oh man this is already useful today and if you think about the fact that they're getting twice as good every year.
[51:42] You know we're very close to visual search being a super powerful tool and that's going to eliminate a lot of the friction we see in stores where people try to get you to, use NFC tags or scan QR codes or do things like that like imagine a future when you just hold your phone up to the Isle in a store, in the phone sees every product on that aisle and it visually recognizes all of them and maybe it even reads the price tag off the shelf for each one of them and it can you know instantly highlight for you, what's a good deals are in that store and what you'd be better off buying from an e-commerce site at home.
Scot: [52:18] In an one thing that's interesting about this is part of the Renaissance on the visual side is it's tied to video games so you know as as people's demands for video games of higher the. The game processors that they're these high-end floating-point machines I've got more sophisticated than ends up that's a great platform for vision in.
[52:42] Incognitive I think but I hear it more used for the vision stuff. So it's interesting is now is part of like AWS Amazon's leasing out you can actually lease out gpus Witcher game processing units, I end up as the cost of those is come down it's it's made this video stuff get even smarter so there's a hardware part of this that's pretty neat so this stuff is. You're not only is the machine getting smarter because the amount of data is going up but there's also Moore's law on the back end helping it as well.
Jason: [53:10] Absolutely and the kind of math that all of these machine learning models use is the kind of math that that I think technically their Graphics processing units not game processing but, yeah but their primary you were right they were invented for games for sure in our friends at Nvidia like being a prime example the, that that the kind of math that those chips are good at is the kind of math the machine learning uses and so that is one of the gating factors for machine learning getting better is having access to big, server Farms of these gpus and and to your point all the big vendors of of cloud computing you know that's the new Battleground is, you know not not CPUs and cores but the gpus. And what's that that's really enable to Renaissance in machine learning that these academics can now rent. Like these these amazing supercomputers for short periods of time to run their experiments and refine their models. Another one that that's super common in this is like a classic example of. Making an existing technology better as opposed to enabling a new capability like visual searches enabling a new capability but obviously a core function of every e-commerce engine is it search function and, the, that search has gone incrementally better every year but it's a largely gotten better because we put better data into the search so we put more attributes into the surge the.
[54:41] The underlying technology for deciding which product is most relevant to which user hasn't changed a heck of a lot in the last five or 10 years and machine learning is now like the first big incremental Improvement to, to search in a long time and the way to think about this is the results, of search that are most relevant to you based on all your past purchases and behavior are probably different than the search results that are most relevant to me, and so using machine learning they can say hey what's every search result I've given to every customer, and which ones are the search results had successful purchase experiences in the ends and which ones didn't, and what did the customers look like that had those successful purchase results and now you know we can personalize search much more to each use or, based on everything we know about them and make search much more effective and relevant that it's ever been before.
[55:42] But that's a classic one we've always had search engines now every search vendor saying their search engine is machine learning based, and that really you know you can't just look at that label and say oh that's the new search engine to get like what you really need to do is test the search engine and make sure it's going to work, better for your audience and with your your product catalog in that that goes double for this next category, recommendation engines so you know we can take recommendation engines for granted at this point like there's so many out there and and you know for a while there's been kind of parody of these, these recommendation engines but but you know it don't lose sight of how powerful these things are, you know a few years ago we saw some data that 75% of all the views on Netflix were driven by product recommendations and this is pretty old now but back in 2013 there was a leak, that 35% of all the revenue from Amazon came from the those product recommendation tiles and I believe the, the plug recommendation tiles and Amazon emails were even higher converting than the ones on the product detail pages and so recommendations are super important and of course using machine learning, you it should be no surprise for listeners at this point you can make recommendations much more personalized and effective for each customer, then sort of static rule-based recommendation engines that are that are kind of the norm that are out there now.
[57:16] So that I'm going to go a little faster cuz I know we're going to come up on time here pretty quick, Annette's category that super interesting in the apparel business returns or are crushing cost, and most of the returns are result of fitment issues so something wasn't the size you expected or didn't fit the way you wanted and Source turn to see machine learning get used, 4 to solve fitment problem so in the old world Zappos tighter going to buy two sizes of shoes that guaranteed one was coming back in that was super expensive so now what you want to do is use data about all the attributes, to accurately recommend the size and maybe even remind a customer that they bought another size before and it fit better in the order they bought this size before and had to return it, tell people get the right size the first time, and avoid buying stuff you're also seeing this get tied in with visual search where you're actually using the camera to help measure the size of the customer and then match that to fitment data tell them by the right stuff so watts of stuff, in machine learning happening around fitment and return avoidance, the whole General filled the personalization this is really hard to shop for right now cuz every vendor is hyping all kinds of new new artificial intelligence and machine learning, capabilities and it's really hard to separate the hype from the reality with all those products but it certainly is true that machine learning, generates more personalized experiences and so there's tons and tons of new vendors out there in that space you know folks they're still in loyalty programs and retention programs are those can be dramatically improved by Machine learning so we're starting to see the first generation of machine learning based loyalty programs.
[59:01] And that's kind of the the main use cases that we talked about right now and in customer engagement.
Scot: [59:08] Got it okay so does summarize what got three buckets Insight generation and that's kind of.
[59:16] I think of that is like next Generation analytics so so analytics that not just Splats data but comes up with insights, business acceleration and that was things that that help you save money improve your forecasting pricing in even practice on, in the customer engagement which are the more forward front-office things are going to improve the user experience learn more about your customers and give him a better experience, people are interested in this soap so first of all. Maybe lay out a lil road map so so a listener is a omni-channel retailer Dave. Get out there like a lot going on in their world right now where does this fall into part ization and where are some places they can nibble and then where do you recommend they go for more information.
Jason: [1:00:04] Yeah so in terms of what your focus should be like you know my high-level advised is ignore the labels don't go look for an AI product but instead look at the list that we just gave you and will put it in the show notes and say, which of those things do we feel like we're most efficient at and we're leaving the most money on the table like art, are you know we not making good decisions about who our audiences are and how we should Target our advertising or we not making good decisions about our pricing or we're paying too much for fraud, or are we in or not doing a good enough job of helping customers discover the right products and add more cards to the more more skews to their cards, and you know our returns to high-dose or two things and so focus on your biggest pain points, and then say alright you know what Solutions out there are using machine learning Technologies to best address that pain point so that would be my sort of, high-level advice and then in terms of specific vendors if you're going to build your own solution, they're they're sort of a 4 horse race for the underlying Technologies for all of these machine learning capabilities and as you sort of alluded to in your experience at spiffy they, none of these vendors require you to be data scientist anymore so they all are like pretty easy tools you probably still have to be a programmer cuz these are mostly api's that you rent, but it's really a 4 horse race it's four of the big vendors, all have these big stacks of AI capabilities that you can rent by the drink in there the really inexpensive in you.
[1:01:41] Add them to your own product so if you're going to hire your own programmer to develop any of these experiences that the first one is IBM with their Watson technology and will put links to all four of these, these Platforms in the show notes one that people don't necessarily think of but is hugely competitive in the spaces Google and they they have the division called Google Cloud, platform services and they they have a bunch of api's for machine learning, they actually invented one of the underlying machine learning models called tensorflow and they've open-sourced it so they they have a lot of great tensorflow Solutions on a gcp but you'll also find other vendors now offering tensorflow because it's become so popular, Microsoft has a complete set of cognitive api's under the there as your services and then as you mentioned, Amazon has a complete set of AI capabilities that are part of AWS and. You know they're all kind of analogous like you'll find basically the same set of api's from all of them you'll find a, a computer vision Library you'll find a sentiment Library you'll find a natural language processing Library a text to speech Library you'll find all these these Legos of machine learning capabilities that you snap together yourself one that's kind of fun that I will highlight specifically for Amazon, is one of their Legos is called the destiny and it's spelled, goofy it's dsstne and that's the actual product recommendation engine from the Amazon.
[1:03:14] Website that they added to AWS last year so you you can actually use, the very system that Amazon juzang and we mention this network effect, it's a huge advantage to be able to get a recommendation engine that's trained by Amazon already cuz you're benefiting from their network of fat. So that that's pretty interesting and then I would highlight that there's some more Niche vendors, there are vendors that as opposed to giving you a low-level API have sort of crafted complete machine language capabilities. Specific for Commerce vendors and so again I'll put them in the show notes but six that come up a lot there's a company called Wiggle, it has one of these machine language based search engines that we talked about in the customer experience portion, there's a company called sentient AI, that has really powerful visual search capability they also have some pretty interesting personalization recommendation engines, there's a company called clarify that that has visual search including video which is super interesting if you're someone that, produces a lot of content on YouTube, the we mentioned the robots that take pictures of the shelves so that's a company called simbi Robotics and they have the, tally robot that dumb Target is testing there's another company at a Stanford called focal systems that have the.
[1:04:45] Computer vision library for doing inventory and then luminoso is one of the companies that has a specific machine language based analytics platform for Commerce and so, there's there's many many more vendors out there but those are sort of six interesting ones to look at to get you started.
Scot: [1:05:03] Yes it's it seems like. And I don't mean to hang up on this but when when I was in e-commerce person just got thinking about our listeners here and I used to buy a solution know you would look at kind of feature benefit. Kind of analysis and cost and all that kind of stuff it seems like machine learning in some of these things add this like other dimension that's only important which is data so says you look at these Solutions. Yeah it's really important to understand. Is this going to operate just on my data is is my data going to be enough to really get a big get big enough bang for the buck cuz I think we're going to see his new models where you're sharing data with other people on a platform. But then you also need to be pretty cognizant about that because this is really important and watch will property you have and once it gets into these Platforms in his learned. Even when you leave and take your data the learning stay so it's really interesting kind of a way to think about things you kind of. As a user of vendor you want a lot of access to data but then you almost don't want your date at a time be.
[1:06:07] In there in the system to use example let's say your. I don't know sorry blaktroniks train the system on on whatever recommendation Electronics now you switch vendors will that. Now competitor switch is there did you start using that now they've they've got a solution to spend trained on your data. Also you know it's pretty interesting that you mentioned Amazon example where it comes kind of pre learned if you will I don't know if that's the right. Verbage pre-trained maybe better in any advice on how people should think about that element of these Solutions.
Jason: [1:06:43] Will know I think you have hit the nail on the head it's the wild west time Electro property and so your, you're exactly right like you have you know because these are almost all cloud-based Solutions you load your data in it you train it you get to take your data back there's it's very clear that the date is owned by you but when you leave you're leaving that model smarter than you found it and your competitors can potentially benefit from that right and so that, that is certainly one issue. But for sure like when you are thinking about areas that you want to invest in and maybe you know the the first area that you want to tackle from a. Sort of testing out a machine warning capability like like one of the big drivers of of the feature that's going to add the most value to you is the one where you have the most data or the most differentiated data so anywhere. You have a data Advantage versus your competitors or versus the market that's a great place to look at 4.
[1:07:44] Accelerating with machine learning and that's the big. Pro in the con of the bill that yourself with these underlying platforms like IBM in Google versus buying a product Eyes solution. Like at wiggler senscient is you know you need a lot less programmers there's a lot less investment to get a customer experience working out of these complete Solutions, but they are going to learn from your data and you know your competitors are going to be able to benefit from that if you really feel like you have some, differentiated competitive Advantage data then that's a good reason to potentially roll-your-own solution using the the the more low-level machine language libraries from from ad the big providers because, they're they're leaving less of that running behind for the next guy.
Scot: [1:08:32] Yeah it seems like it'd be smart to go make a coalition with people that aren't competitors and say hey let's pool our data kind of create our own data pool and.
[1:08:43] I almost wonder if there's like some way to do something little Pine Sky almost like a Dana Co-op.
Jason: [1:08:49] No it's.
Scot: [1:08:50] Having your own pool of your data isn't as hopefully you still need that other data but you still want control over the day this it's kind of a really interesting turkey challenge.
Jason: [1:08:58] There's models out there right now so an interesting one is, Adobe for device detection right so you don't Scot you and I on a bunch of different devices they all have different cookies on them so it's hard to tell when you visit a site on your tablet and then later on your smartphone that you're the same user, Facebook and Google know because you've authenticated yourself, in in the on all those devices on Google and so Google recognizes you across all those different devices so Google has a big advantage over most e-commerce sites and Facebook has a big advantage over most e-commerce sites, in terms of recognizing each user and so you know folks like Adobe have literally set up a device data Co-op. So that multiple websites can share, what they know about which devices you own and your Anonymous so your Pi isn't in there but when someone you know gets any device that, did Scott Wingo owns they can go to this Co-op and find out what the ideas of all the other devices that Scot own so that they can, do the Multi-Device attribution and so we've we've seen it there we are starting to see these kind of data, Co-op submerged in a couple places I think in some cases even competitors collaborating so for example a lot of the insurance companies, they have to pay claims when a natural disaster takes out a roof, they're all sharing their data their photo libraries of all the roofs that they've had to repair and what the level of damage was and so now they have this big data repository that they all benefit from.
[1:10:37] When the hurricane strikes they fly a drone over the neighborhood of takes a picture of all the roofs, add a machine learning algorithm you know tells you in half an hour how much is it going to cost to fix everyone's roof in that neighborhood so so pretty cool stuff.
[1:10:51] And Scot afraid that is going to be a good place to leave it because it is happen again we've wasted a perfectly good hour and tax of our listeners time, listeners as always we certainly want to continue the dialogue so please visit our Facebook page if you have any questions or comments interview like today show we would greatly appreciate a review on iTunes.
Scot: [1:11:14] Thanks for listening you're ruining maybe someday this entire podcast will be automated intelligence artificial intelligence.
Jason: [1:11:22] Or maybe it already is? And with that, happy commercing!
116 episodes available. A new episode about every 7 days averaging 58 mins duration .