In case you missed the memo, Microsoft loves ChatGPT and is all-in on spreading the natural language processing AI across all of its services. However, Microsoft also sees the benefit of the AI functioning as an instructor for human-to-robot interactions. That is why the company is now leveraging ChatGPT to code actions for drones and robots.
Microsoft says it has been researching whether “ChatGPT can think beyond text, and reason about the physical world to help with robotics tasks.”
Essentially, the company wants to know if OpenAI's chatbot can instruct robots without needing to learn the relative programming languages. Furthermore, the research looks into whether ChatGPT can create instructions without an underlying knowledge of robotic systems.
“The key challenge here is teaching ChatGPT how to solve problems considering the laws of physics, the context of the operating environment, and how the robot's physical actions can change the state of the world,” says the team from Microsoft Autonomous Systems and Robotics Research.
The research was done mostly using Python as the programming language. Robotic scenarios the team attempted to get ChatGPT to do included code generation and zero-shot planning. To help, the AI was given access to object-distance data and object-detection via application interfaces.
Code Capabilities
While ChatGPT can create code from thin air, it can generate code because it was trained on datasets of massive amounts of code and text. In other words, the AI can take code that was written by third-party coders. And yes, ChatGPT is tapping into that code without asking permission for the people who wrote it.
It leverages Codex, the GPT-3 engine feature that was developed by OpenAI and Microsoft. This is the same technology that underpins GitHub Copilot, another tool that “borrows” the work of others and has found itself in legal trouble over copyright.
In the research, Microsoft used ChatGPT as an interface for a drone. The company says there is “a potentially more versatile tool for the robotics domain, as it incorporates the strengths of natural linage and code generation models along with the flexibility of dialogue.”
“ChatGPT asked clarification questions when the user's instructions were ambiguous, and wrote complex code structures for the drone such as a zig-zag pattern to visually inspect shelves.”
Tip of the day: Do you know that Windows 11 / Windows 10 allows creating PDFs from basically any app with printing support? In our tutorial, we show you how this works via Microsoft Print to PDF and Bullzip PDF Printer to save a PDF from any app, even with advanced options like adjusted quality, multi-page printing, and password protection.
