July 9, 2024
Updated Video Edits, Links, Transcripts, and Much More Information Eminent… -CSK
Global Tactile’s Creator & President Doctor Christopher S. Kallie had the honor of being interviewed by Hellen Keller Prize Laureate, Distinguished McKnight University Professor, Director of The Minnesota Laboratory for Low-Vision Research, and former Chair of The Department of Psychology at the University of Minnesota, Professor Gordon E. Legge.
I would like to personally thank Dr. Legge for his generous support in my professional career and on this project. -Chris Kallie
Audio Interview with Professor Legge (Part 1)
Insert Audio Here
Video Interview with Professor Legge (Part 1)
Transcript of Interview with Professor Legge (Part 1)
Chris: Hello, Gordon!
Gordon: Hi Chris, how are you?
Chris: Good! How are you doing?
Gordon: Pretty good.
Chris: You’re sounding young as ever.
Gordon: Yeah, well the years fly by.
Chris: I know, isn’t it something.
Gordon: So, are you physically in Wisconsin, in Milwaukee area, or where are you these days?
Chris: I’m back in my hometown area in the suburbs of Milwaukee. There’s a little town called Hartland with no “E”, it’s HARTLAND, and that’s where the company is. And I live where I went to middle school and high school in Oconomowoc.
Gordon: OK, Good! So, I wanted to catch up. I’ve sort of lost track of your work with Cybernetics, Tactile Controllers, Luna Navigator. I know you’ve branched out quite a bit from the original NIH project, probably, but it would be great to catch up a little bit.
Chris: We have made huge progress over the last two years. And remember the fundamentals are the same. The thing is: Emergent Properties and Metasystems. So, all I am telling you is everything I promised to you and everybody else, and then all of the sudden, all of the cool, fun stuff that comes out of that, and some pretty kind of scary stuff too. Like, if you can control a machine that carefully, then you can control a deadly machine that carefully, or you can control a life-giving machine that carefully. So, I’m giving examples of both. And I think it’s fair and important to do that. And I hope you agree, but if you disagree, you are the only person who I might even listen to. So, your voice is important. And I’m going to give you the rundown.
Gordon: OK, great.
Chris: I think the best way to describe the system is to know your audience and my audience right now is somebody who likes playing baseball. So we’re gonna play a baseball game and I’ll walk you through it with the Luna Navigator. Not only playing it, now, but also scoring the game which is really exciting and that’s where the Metasystem comes in to play. So the first thing you would do is launch the App, select a Map. In that case you would select either your offense or defensive map or your play position or whatever map you need. Let’s select a batter’s map. So a batter would select the batter’s map and then the first thing you probably would want to do is select the pitcher’s mound. And then you’d walk to the mound.
Gordon: OK, can I chime in with a question?
Chris: Yeah, absolutely!
Gordon: Are we fundamentally talking about a video game here? Is it on a PC, a Mac, smartphone, or what?
Chris: Right, it’s the Luna Navigator, and it’s available for free on the iPhone App Store. So it’s free in the US and it’s five dollars and every other country. So, let me walk you through the fundamentals of the Luna Navigator, because, even though it’s complex at first, it’s sort of like everything else: once you get it, it’s like wow that’s cool. So, if you download and install the App what you get is: it does two fundamental things: Number one it uses the Haversine Formula to compute Geodesics. That was the original thing that I came up with back in 2012 when we were doing the Veering Experiment. I’m like, we have all these GPS’s, technology is going to get better. What we need to do is compute the crows flight distance from where you are, and then you chop it up as little as you need, and if you have an obstacle, you can do stuff like add video to push around it. So we haven’t added video yet, but the Luna Navigator does the following: the first thing it does is it computes your orientation and location. It computes your heading, latitude, and longitude.
Gordon: How do you give it the heading? How does it know the heading?
Chris: In this particular case, we are relying heavily on Apple, and Apple is our application vendor, and we are using their sensors.
Gordon: OK, so it’s the way the phone is oriented then, would determine the heading.
Chris: Yes, the phone is everything. And actually, there are three geometric algorithms, and three biometric algorithms which (trade secret) but, it’s a Kalman Filter that filters out your body motion a little. And it works quite well. So, the heading, latitude, and longitude are computed by the position of the phone. Now, if the phone is on your chest, because we have a Chest Harness you can buy, or you can make one, then it’s in biometric mode. And so what it does is it filters out your pivoting-tilting motion as you jog, stand, run, or if the harness is on crooked, it just takes out that error. And it works awesome. Now the fundamental position is face up. And that’s called “Geometric 1”. And then, you can literally use the phone, actually you can use the phone in all these orientations. You can use a line of sight on the phone. In fact, it wouldn’t be outside of a crazy act to add a scope to this thing, and slap it to the side of a scoping machine. And actually we’re already doing that but will get to that advanced level later. So, in Geometric Mode if I have it on, let’s say, on my console of my armchair, or if I have it on a boat or a car, if it’s in “Geometric 1”, where it points is where it’s trying to compute.
Gordon: Ok, makes sense.
Chris: And then you can hold it on its left or right side and that adds biometric filters and they’re really cool and I’ll tell you about those later. So, the heading, latitude, and longitude, are computed by two basic things, now (more in the future): the first thing is GPS. So we know the error, one standard deviation error is about 3 meters on a good day. It’s good for anywhere out in nature where I’ve been. It’ll get you close. Then the Heading is using a Compass that is provided by Apple. So those are the first three numbers. Now the second three numbers are your pointer, distance, and bearing. So where do those come from? Well, they’re actually they’re sort of like you’re heading, latitude, and longitude, in a way, because what they are doing is they’re pointing to the next waypoint. So, what it does is when you launch Luna, it just points towards Global Tactile. So, right now if you wanna know where I am exactly, I’m bearing XXX degrees and I’m XXX kilometers away from Global Tactile. So now you know exactly where I am by a vector that is provided by the map and the default map is just the North Pole of the Cybernetic Universe: Downtown Hartland. So, then what you can do is load a map and that’s one easy button, and I’m going to load, why don’t I load, oh, here we go: Pine Cove race course. So that’s a race course that we used in our front yard that actually, now this is preview information but it’s important: We actually put the Neo Controller inside of an RC car and it drives along the course. Now, that brings us to a very important discovery or rediscovery in my opinion: cybernetics is all about steering. Steering is all about a vector, which is magnitude and direction. Those two numbers are what you need. And that’s what the Neo Controller delivers by the way. So we’ll get to that soon. So, we load a map and now it says Race Course 1.
Gordon: Loading a map, if I’m a user, could I key in my home address, or could I create a point of interest somewhere in my vicinity?
Chris: Yes, very good question. Fundamentally important. We use JSON Files. If you know the latitude and longitude, then you can give it a label and you’re done. You can make a map on your own. I teach everybody how to do it. Everything I’m telling you now is on the website I’m trying to organize and improve. The JSON file is very simple. Let me open one up and describe it to you visually. OK, here’s Pine Cove Race Course. So, there’s only 4 routes in this course. So, I gave them ID’s 8001 through 8004. No big deal. So, each one has an ID or number and that’s how you can select them. So, these are 8001, 8002, 8003, and 8004. And then the second part of the JSON file is the route, so the route names are Race Course 01, Race Course 02, Race Course 03, and Race Course 04. And then the last part is the labels. Oh, excuse me, I skipped the waypoints. Waypoints are strictly a list of waypoints. There’s 5 waypoints in this race course.
Gordon: OK, so those are lat-long coordinates?
Chris: You got it. Yeah, and right now we’re using 6 or digits of precision, or 7, beyond the decimal. And that get’s you down to like a meter or so.
Gordon: But right now creating these maps, these JSON files, is a techie?
Chris: Well, it’s copy-paste, but it is readible. You have to get the format right, if you have a comma or a space in the wrong place, it’ll dump. And, so what you do is, I provide as many maps as I can online. I provide a map of a blindfolded or non-blindfolded. You don’t have to be blindfolded to use these, you can use these for Geocaching. I am using it for fishing and hunting. So, I’m trying to make something universally accessible. So, I am using it for Archery, I’m using it for Guiding a Canoe. I’m trying to build a sailboat but we need more money, that actually physically guides itself through a course. So, we’re going very far and wide, I’m doing driving studies every day now, we analyze the data at the, you know, I make gigabytes of data with this machine, which you can too right now. You can buy Luna, or not buy it, it’s free. You can download Luna, hit “Record” and drive home and it’ll make a map of your course home.
Gordon: Oh, OK, well that’s interesting. Could I walk around on my property and make a map?
Chris: 110% Yes! Yes! Yes! You can. You can. It will do the accuracy that it can in real-time, and it records 23 independent variables 10 times per second. And so all those variables get recorded in a CSV file that’s easily readable and in fact you can even use, oh, I forget the name of it, Numbers on your iPhone, and you can plot your course. So, I’ve been mowing the lawn with it. We’re going to do studies to see how good your overlap of the lawn deck is, how good your efficiency is.
Gordon: I like that.
Chris: If you have an incident while you’re driving, whether it be a lawnmower or a car, this device proves whether or not you were even touching your phone.
Gordon: So, theoretically, you could build a self-, a robotic lawnmower using the routes for your particular patch of grass.
Chris: Yes, and we’ve already proved it. And there’s even video proof of it: Last February we ran courses out on the ice that were actually originally blindfolded canoeing, and blindfolded cross-country skiing courses. So, I’ve run the same course with just about every machine I can get out on the lake. And, we also have.. Since you’re asking fun and cool questions. I have a Neo Controller that’s a machine-level one. It’s 10 times stronger than the one that’s handheld. So, it’s very dangerous and it goes inside a wodden box, and then you can attach it to things, and I attached it to home-made cabling on a canoe, which I’m also attaching a sail to, and side- they’re called daggerboards, so basically I turned the canoe into a vehicle that will be controlled completely by this machinary.
Gordon: OK, that’s interesting.
Chris: Yeah. So, OK, let me see: There’s 1, 2, 3, .. and the 4th item inside the JSON file is the labels. So, in this case, the labels for my little race course are: “Start”, “First Corner”, “Second Corner”, “Third Corner”, and then “Start”. And so, let’s go back to the baseball game now, so you get the idea. I’m going to just load my Nixon Park Baseball File, which you can download for free at Global Tactile. ID 0 is the pitcher’s mound, and that’s at 43.10…, -88.34…. That’s the pitcher’s mound at Nixon Park in downtown Hartland. And then first base, and then second base, and third base, and home plate. So that’s IDs 0 through 4. Then I have just two routes. So, 1 through 4, they’re technically routes, but they’re only single waypoints, so you just go there and you’re there. But then ID 10 is a home run, which includes..
Gordon: The routes are running to first, second, third, and home, is that what the routes are?
Chris: Yeah. And the pitcher’s mound is ID 0. And of course you can make your baseball game anyway you want to, I just made it that way just to teach people. So, let’s say we all want to meet. I send, I either drop-box this or I can actually text it to you. It’s a very simple JSON file. And I can be like “Hey, see you at 10 on Saturday. Meet at the pitcher’s mound”. You get that file, and then you launch Luna and load that map and it points to the pitcher’s mound.
Gordon: I see. But what points in what format?
Chris: OK, repeat your question. It was a very important one. Basically how to use it?
Gordon: Yeah, I’m just imagining, let’s say, you’re up at the cabin. I could create a route from my dock up the path to the cabin, or something like that. I could have like 3 waypoints. Could I literally, with the software that I would download from Apple, record that kind of a route now, when I download the app?
Chris: Yes. So, very, very good question. And I hope I get this exactly right: There are two ways to use the recording apparatus. Number one, if you’re anywhere and you want to record exactly what that phone is doing, press “Record”. You are literally making a complete map at 10 Hertz, with additional information like accelerometers, and gyros, and magnetometers.
Gordon: You mean, even if I start recording and walk along the path, it’s recording a sequence? It’s not just a single, static point?
Chris: Correct. At 10 Hertz it’s recording every motion of that iPhone. For functional, and I’m talking High-Speed, Downhill, Blindfold Skiing, I can use it at 10 Hertz to follow an imaginary or plotted course that I previously plotted.
Gordon: Let’s say I started recording and I walk down the path and I want to put a waypoint. I want to stop recording and I want this to be my waypoint.
Chris: There’s numerous ways to do that. The easiest way is to press “Stop” on the Recorder and press “Record” on the Recorder again. Because, what that does is it makes a new file. So, if you’re connected to the cloud, or if you’re connected locally, either way the story is the same: that thing will keep updating at 10 Hertz. When you start recording, that original time stamp is the name of the file. When you stop recording, or when you run out of batteries, or when Luna crashes — which it will never crash because I wrote it — but if Apple crashes or if somebody else crashes, then it will still record up to the last 10th of a second.
Gordon: Yeah, but what if I want a series of points on my route? I’m going to have, like, 3 or 4 separate recordings now.
Chris: There’s two ways to do it. Number one, just press pause and play on the recorder, again, and that will divide up your CSV file at that time stamp. That’s one way. If you want to keep recording, then grab your iPhone and make a gesture. That’s really cool. Now, we don’t have the software yet to analyze gestures, I’m just telling you that’s just how it’s going to work, and I’m telling you it’s awesome, and it’s going to work great. So, what I have been doing, I haven’t analyzed the data yet, but it’s on my highest priority, is, you can just shake the iPhone and now if you know the X- Y- and Z-coordinates of you iPhone you can shake or rotate the iPhone and make a gesture of your own choosing, and anytime you repeat that gesture, it will be your own personal code that that is what you wanted it to do.
Gordon: Ok, but I’m still trying to puzzle. Let’s say I want to create a route like you were describing, either for the canoe or the baseball bases. Let’s say I want a route that goes Home to First to Second to Third. So, I start recording at Home, I walk to First Base, do I hit Pause or what do I do?
Chris: You just take a screencap of your Luna Navigator, and then you can then take a photograph of the location. So, before I had the recording device, which has only come out about a month ago, I was using screencaps and relying on them heavily, and taking personal notes. Usually what I did is a screencap, followed by a photograph, or vice-versa.
Gordon: But I don’t think that’s answering my question. Like you said at the race track, you have a map that has those 4 waypoints.
Chris: Yeah, the way to get the waypoints is either: take a screencap of the Luna Navigator because it displays your current latitude and longitude to 7 digits.
Gordon: Oh, OK. So it’s not going to build the map point-by-point?
Chris: Not yet.
Gordon: I have to feed in those coordinates?
Chris: Yeah. We have not fully-automated every luxury. That’s a very fair question. But for an essential, minimal viable product, I did not build that part in yet. If you think something should be on a high priority, I am always open to suggestion. So, we have been upgrading, and if something feels like it needs more attention, let me know.
Gordon: Well, it would seem to me, superficially, that, OK, let’s say I want to build a route, I’ll say around my cabin property, but let’s say it could be out in the woods or even on the lake for example. I want to go from, OK, I’m starting recording at point 0 and when I get to point 1, I just want to push a button and it’s going to record in my map file that latitude and longitude and maybe I guess orientation. And then I just want to resume. And, I want to go to point 2, 3, 4, I just want to, with a click, I want to be able to feed them in.
Chris: Yes. So, a screencap in that orientation, or pressing pause and play again on the record button will do both of those effectively, because if I press my two buttons: I’ve just taken a screencap; now if I press record and I stop pressing record, I also have about a 1-second log in CSV format.
Gordon: So, let me go back, I think, like where this project kind of started. Let’s say for a visually-impaired user. So, wherever I am now, I want to get to a particular target waypoint, is it easy for me to load that waypoint and then have it give me direction and distance feedback to get to that waypoint?
Chris: Yes, it’s fully accessible. It uses the VoiceOver feature on your iPhone. And I’ve had blind people use these. So they can load the map no problem, and then on top of the map, you’ve got to load your routes, so there are more buttons, but yes it’s fully accessible. Every one of those buttons: half of them are essential for really getting a profound understanding, and the other half are for higher performance. So, any other questions?
Gordon: Well, I’m still trying to puzzle. Say, there’s an important waypoint. Let’s say I’m out boating, and I want to get back to my dock, let’s say. How do I tell it my target is the end of my dock? And how do I get it to get me back there.
Chris: Yes, you just load your local map. I’m going to load my “Home to Work” driving route, and my nearest route now is 22 meters bearing -88 degrees, so that means I’m facing south. So, as soon as you load a map, the very first route on the map gets loaded, and replaces the “Global Tactile” vector of pointer, distance, and bearing. And if you don’t close the app, then whatever map you left on is the one that’s there. If you close the app and then reopen it, you just have to reload your favorite map, or the most important one, or your current escape map. So, I’m going to tell you an interesting story that sort of made me realize that this is more than just a thing for blind people. I was unfortunately single at the time, and I was hanging out at a bar at like 5pm, but it was winter, and I snow skied, I cross-country skied across Okauchee Lake, which is a very big, it’s like 1,100 acres, not huge. It’s 1/10th, you know, it’s not Minnetonka big, but it’s shaped like Minnetonka: there’s a lot of bays, and fingers, and channels. So, of course, I know the lake, I grew up playing on that lake. But, I got sick while I had a glass of wine, and then I had another, and I think somebody put something in it. I don’t know, but I got sick, and I was like “I need to get home!” Well, I had the machine on, and I’m like: “Point Home!” So, what I did is — I did get sick a few times on the way home and had to stop — but then, when I got into the open lake it pointed directly towards my apartment. Which shaved an entire half an hour of travel time off of my route.
Gordon: Yeah, Good!
Chris: Yeah, and I didn’t have the recorder running, but I do, actually had another GPS in my backpack because I was still building the Luna Navigator. So, I do have a recording of that route, and it shows me perfectly going straight to my house. And then, I mean, I’m going to say, it was trespassing through the neighbor’s house. But I wouldn’t have known where the shortcut was, and I would have either gone too far north or south. I went straight to my apartment, I shaved a full half hour out of going through the public access and taken the road home, at like 6-7 pm, but it was winter and it was dark. And I was thanking my lucky stars and that was my pivotal moment where I’m like “this thing is not just for blind people”.
Gordon: Well, I’ve often thought.. I’ve never really been out totally alone in a boat on the lake, but I’ve always thought, OK, if I’m going to do this, I really need to have a beacon that will get me back to my dock.
Chris: Yeah, this will do that for you.
Gordon: As long as I don’t try to go straight through the island.
Chris: Yeah, so that’s where you need a route.
Gordon: Ok, I just want to mention something just before I forget it, because it seems sort of relevant. I don’t know if you ever listen to the New York Times Daily Podcast, but they had one today on robotic, actually weapons, being developed in the Ukraine. So, basically they were making the point that, with basically home electronics, these guys in Ukraine are building very sophisticated drones and other weaponry to use against the Russians.
Chris: Yeah. And it’s very easy to do that with this technology. Gordon, the hardest thing for me… When I joined the Lab, I thought it was going to be far away from the danger zone, but I found myself smack dab in the middle.