This section lists all blog posts, regardless of topic.
CES 2015January 9, 2015
January is an interesting month in the tech world because of CES -- lots of shiny gadgets being shown off, and the direction of technology becoming a bit clearer. Here are some thoughts on the next 15 years of tech based on this new information.
Perception & Devices That Move / Interact
More and more it is becoming apparent that the perceptual abilities of technologies are sharply on the rise -- the ability to devices to sense and perceive the 3D world around them. This dovetails with recent advances in machine learning.
The intuition, then, is that over the next 15 years we will see significantly more devices that can:
a) | Accurately sense/perceive the physical environment around them. |
| b) | Therefore, interact appropriately with the physical environment around them. |
This is a big deal. In a sense, what we're talking about are "robots", the old concept of these mechanical creatures walking around and doing things. But this is a broader vision, really -- it is devices of all kinds, small and large, being able to perceive the real world, and behave in it. Cars, drones, vacuums, doors, thermostats, TVs, garage doors, etc.
What will this ultimately look like in 15 years? I still expect the most noticeable thing will relate to autonomous driving, but be ready to see this theme all over the place.
In industry, I see this as being incredibly significant. Automation has been kept back by this perception problem, and so if it is significantly solved, expect significant developments in industry, and in some cases, replacing person-powered jobs in factories (etc) with intelligent systems.
The Size of Computers
This shouldn't be surprising at all -- more just a confirmation of the trend we've seen over the last 40 years continuing. Computers can now be fit inside of something the size of a shirt button. So what does this mean for the next 15 years?
I think it's just "computation everywhere", which again isn't a new idea, but I think there will be a synergy that could be profound between these small devices and the Internet. It's the whole IoT thing -- Internet of things. As I've commented on in the near past, these tiny computers are really the "eyes/ears" of the "Internet organism".
I'm not sure if this will really become apparent in the next 15 years, but it could -- the increase sense of just how much this Internet organism can sense and perceive and thus potentially act on. My expectation in 15 years is that there will be a big up-tick that way.
Just expect lots of "smart" stuff... smarter street lights, smarter homes, smarter appliances, smarter cars, smarter medical diagnostic devices, smarter clothing perhaps (ex. shoes that count steps, etc.), etc. etc.
Screens / TV
Last but not least, and somewhat of a surprise to me (but perhaps shouldn't be), still lots of energy being put into the development of screens. 4K, talk of high dynamic range, etc. This all points to the next 15 years continuing to be a rich time for displays, increasing resolutions, increasing contrast, dynamic range, etc, etc. I still think this will slow down at some point, because we're approaching diminishing returns at some point, but for now, the engine is still churning pretty hard.
Function Search Engine / How To Search EngineDecember 28, 2014
I had an idea recently that has sparked a lot of excitement in my mind... not infrequently, I find myself needing a function that I know has probably been implemented 10,000 different times by other software developers over the ages, and I groan that here I am, about to do it again. Other times the function might be more unique, but I'm still convinced that its something that other people have already implemented at least a few times.
What if there was a really compelling way for developers to share functions/classes? There already exist places like GitHub for sharing
projects, and that's great, but so often what I want isn't an existing project, it's an existing function. Imagine this search engine was more than a search engine, it was a combination of a search engine and a source code repository like GitHub. People could upload functions, associate unit tests with them, and the site would allow others to view those functions and even run them interactively on the website to play with them. Users could file bugs against those functions, resolve those bugs / commit fixes, etc. Users could comment on functions or make requests for improvements, etc.
The second ingredient would be to make these functions very findable through clever use of NLP, etc. So, you're coding in Java and want code to download a webpage? No problem -- search for "download webpage", and see functions that do exactly that.
This could be expanded to also encompass the idea of "how to's" / "example code". Search for "Java convert string to integer" and a function would be found that demonstrates how to do that. You might not want to use the function in the end, but at least to be able to examine the function to learn how to do what you were asking for.
Next: The ability to write functions that call other functions on the site. Technically you'd be calling a certain REVISION of that function, since future revisions of the function you were calling might cause breakage, but automated tools could be used figure out if it was safe to call a newer version of that function. It would also be interesting, upon modifying your function, to see if any of the tests for things that call your function break. (A kind of canary in the coal mine that might suggest you broke something in your function)
I think this idea combines (can combine) some powerful things:
 | Open source |
| Crowd sourcing |
| Wiki |
| NLP |
| Exponential nature of technology (building up more powerful pieces) |
| Ratings: People could rate functions to make them higher in the search results |
| Status: As people contribute more, they gain status. Useful for resumes, feeling good about one's contributions, etc. |
Also imagine people defining "interfaces" + tests that define useful bundles of functionality, and being able to associate functions with interfaces. The system might even try to automatically detect when a new function matches an existing interface. Lots of interesting possibilities there.
The system could automatically sort functions by their runtime to show you which implementations were fastest, etc.
Yet another way to search for a function could be to define the arguments of a function you're about to write, the description, and a few sample input/output value pairs, and then have your IDE connect to the service to see if there are any existing matches or close matches. If so, it could essentially fill in the body of the function you were about to write automatically, or download the function and simply refer to its implementation.
Then imagine associating implementations in one language with implementations in another so that if you're learning a new language, you can simply click on a link to see how some particular thing is implemented in another language. It could also then generate metrics to show how many relative lines of code you need to implement something in one language VS another, the speed of execution, etc. That could then be used as a semi-automated tool for converting code from one language to another -- just use the properly mapped function.
And the final idea I'll propose is to provide a way to map what a function does onto a semantic model with the goal of making it even more possible to simply write a specification of what you want done, and have the site be able to cobble together the appropriate functions to do what you want, and spit out the required code.
Lots of really exciting possibilities in this space...
Voice Scripting / Intersecting Circles of Interest / Understanding SelfDecember 6, 2014
Everyone has things that they are strongly interested in and/or enjoy doing. A realization this week is that occasionally we come across an activity that combines 2 or 3 of our interests, and that can result in very strong levels of engagement. A way to visualize this is the venn diagram -- intersecting circles.
I experienced this phenomenon this week as I was setting up some voice-activated scripts/actions on my computer. I have things setup so that when I press Ctrl-Q, my computer starts listening (via Google's web speech API), and I can then issue voice commands. If I say "email John", my work email program creates a compose window and fills in John's email address. If I say "email Meredith using Gmail", a web browser opens a Gmail compose window and fills in Meredith's email address. If I say "lookup John in the company directory", the company directory page opens in my web browser with results filtered for "John". I could go on and on with the plans that I have for this technology... what is interesting is that setting those voice scripts up was mind blowingly compelling for me. I could easily lose myself for many hours, days, perhaps even months, implementing that kind of thing.
As I was setting this up, I recalled my days working for my former employer. Something that made me unique was scripting. I had a directory with 100+ scripts, many of them built on top of each other, allowing me to do all sorts of things with incredible efficiency.
It's no mystery to me that I'm hugely motivated by
efficiency and
automation. Those are two highly overlapping areas of interest. But upon further thought, there are several areas of strong interest at play with these voice scripts. It's fascinating to unpack those things, because it uncovers significant aspects of who I am, the things I value, what I find natural, etc.
| 1. | Automation: Another way to put this is that I find repetitive tasks extremely frustrating. Any time in the past that I've had to do data entry, for example, it's like a "nails on chalkboard" experience. Working on a computer is no stranger to repetitive tasks, or tasks that consist of a bunch of predictable steps that just begs to be simplified down into one step. Any time I'm able to automate something, it is extremely rewarding. See also: Efficiency, Productivity. Creating voice scripts is a powerful form of automation. |
| 2. | Efficiency: I've heard that my grandpa Green highly valued efficiency, and if so, then the apple didn't fall very far from the tree. My mind seems to naturally gravitate to thinking about how to do things more efficiently. As mentioned, there is a strong connection here to Automation. Another connection is that I really dislike waste, and inefficiency is wasted time. Creating voice scripts is a great way to make doing your work more efficient. |
| 3. | Productivity: Strongly related with 'efficiency' and 'automation', I find high levels of productivity extremely rewarding. The todo list with ten items on it, all checked of as completely, is thrilling. Anything that helps me get work done faster is very compelling, and voice scripting fits this bill. |
| 4. | Natural Language Understanding: The area of technology that I'm most interested in (and have been for about 6 years) is the problem of natural language understanding, and this is a key component of creating voice scripts, because it requires understand/mapping natural language commands to the actions you want to achieve. |
| 5. | Voice Commands: Back in 2008, I wrote an application called "Grace" that ran on an iMac in the corner of our kitchen. In a sense, it was very much similar to the voice scripts I'm building now, but was focused on home life rather than work life. Having a computer be able to understand the spoken word was somehow very compelling and exciting for me, and still is. It puts a smile on my face. |
| 6. | User Interfaces: When I was a teenager I discovered that I had a significant interest in graphical user interfaces. I loved the challenge of crafting a UI that would allow a user to get something done. One day I was contemplating my intense interest in natural language understanding, and it suddenly dawned on me that there was actually a connection between my interest in that and graphical user interfaces. That is, they are both user interfaces. Voice scripting is a fascinating addition to keyboard and mouse. My general sense is definitely not to replace keyboard and mouse with voice. I think that's a huge mistake. The idea is to supplement keyboard and mouse with voice, and to do that where voice is more powerful. For example, if I want to email Meredith using Gmail, and if I use keyboard and mouse to do it, I need to: 1) Move my mouse down to the Windows task bar, 2) Click on Chrome, 3) Press Ctrl-T to open a new tab, 4) Click on Gmail, 5) Click on 'Compose', 6) Type in Meredith's email address. That is 5 or 6 steps to initiate the desired action. Using a voice script, I have to: 1) Press Ctrl-Q, 2) Say 'Email Meredith Using Gmail'. The 2-step solution is cognitively much simpler and takes about half of the time. Meanwhile, if I'm typing an email and I want to type something in a bold font, would it be easier to have to press Ctrl-Q and speak "bold font" than it is to simply press Ctrl-B? No, of course not. It's all about choosing the right tool for the right job. |
| 7. | Minimalism: It's no mystery to myself or others that know me well that I seem to be a minimalist by nature. I like clean, tidy environments that aren't overrun by junk. (There was another famous minimalist -- his name was Steve Jobs!) I love well designed things that have had all of the rarely-used knobs hidden away in cupboards, and execute brilliantly on the core functionality. Voice scripts are actually a pretty strong embodiment of this vision. There are no knobs to be bombarded with. You just say what you want, and it's done. What a beautiful minimalist UI! |
| 8. | AI: At some point, as perhaps many technology people can relate to, I became a bit captivated by the idea of "AI". Probably because we all realize how powerful / useful intelligent systems are. Being able to create software that leverages intelligence is extremely compelling to me, and voice scripting intersects this realm. |
| 9. | Cutting Edge: As with many technology people, I get a real kick from seeing and using things that are new and interesting, which push the limits of what is possible. Voice scripting has that flare to it for me. |
| 10. | Creativity: I enjoy my work much, much more if there is a strong element of creativity involved. Envisioning how voice scripting should work, what use cases to target, etc, is a nicely creative process. |
| 11. | Self-directed Work: Probably like most people, I find implementing my own ideas more interesting and rewarding than being told "go do this". |
| 12. | Ideas: I'm an idea person. I find it thrilling to come up with ideas that seem valuable. I've never actually seen someone demonstrate a really effective synergy between keyboard, mouse, and voice, and so in envisioning that possibility, I feel like I'm tapping into a powerful idea. |
So this voice scripting stuff combines many different areas of strong interest for me. I suppose it shouldn't be surprising that setting up voice scripts is such a rewarding endeavor for me. It's such a strong intrigue that I find it hard to stop thinking about it.
The end-goal vision is this: Being able to do one's work incredibly efficiently with powerful voice scripts, which are backed by robust natural language understanding.
Want to start working on a task? Just copy the task ID into your clipboard and say "work on this task" -- your time tracking program will start logging that task, the task will be displayed in Jira, a new directory will be created for the task with the appropriate name, a new Mathematica notebook will be created and placed into that directory and displayed on your screen, the type of task detected from the Jira issue and the appropriate notebook template used so that the various functions you need to perform the debugging (etc) and right there in front of you, etc.
Done working on a task? Just say "resolve the task" and the Jira issue will be resolved, with the appropriate time spent recorded from your time tracking tool. Back in your time tracking tools, your work log will be appended with a line item indicating that the task was completed.
Need to comment on the issue you're working on? "comment on task"
Need to create a new issue? "create issue", "create bug", etc.
Need to chat with someone? "chat with John"
Need to send an email to someone? "email John and Amy"
Need to request a tag move? "request a tag move" and have your recent commit be automatically found and an appropriate email with "To", "Subject", and "Body" filled out. Just review and press "Send".
Need to restart a software component that would normally take a bunch of manual steps? Just say "restart <blah>" and have it all done automatically.
Need to go to a directory? "goto <blah>"
Need to download a new software build? "update build"
My sense is that there are many strong candidates for core / frequent actions to be packaged up nicely, allowing work to be done very quickly.
Part of the thinking here is that the human mind wants to be freed up to do the high level thinking, and not be burdened with endless clicking, navigating, etc. If all of the unnecessary low-level stuff can be factored away, it leaves the mind better able to focus on the problem solving, and hopefully, that results in a higher level of mental engagement, more passion for the work being done, etc.
One final closing thought here is that I work from home, so yabbering away at my computer all day is no problem. This wouldn't really work for people clustered around desks in an office environment. So the utility of this idea more generally might not be as great as I might hope.
older >>
