Header Promo Announcement

Cozmo - SDK - LP

Anki is thrilled to announce the Cozmo SDK Beta. Cozmo is one of the most capable consumer robots available today. Combined with the SDK, Cozmo is exactly the kind of robotics platform we’ve always wanted for hacking and research projects. The software and hardware were designed from the ground up to give developers unprecedented access to robotics. Using only a few lines of code, you can tap into and expand upon many of Cozmo’s robotics systems, including Computer Vision, Path Planning, Manipulation, Animation, and much more.

The first Beta version of the SDK will be available when Cozmo launches this October. This page is a preview of some of the SDK’s most important functionality. Cozmo is available for pre-order today; if you pre-order, you’ll get Cozmo two days before he is released in stores.

What makes the Cozmo SDK unique?

The SDK provides high-level interfaces for performing complex robotics tasks such as face recognition and planning paths around obstacles. For developers who want more control, there is also direct access to low-level functionality, such as driving motors and getting raw camera data, allowing developers to create highly customized behaviors. The broad feature set makes Cozmo accessible to curious tinkerers with enough depth to be used for robotics research. We are excited to share these features of Cozmo with everyone who already loves robotics and those who are eager to learn.

Here is a short Python code example from the Cozmo SDK. It gets Cozmo to look for faces, and every time a face is in his camera view, his backpack LEDs turn blue.

Changing LEDs is an example of a simple low-level function, the kind available in many robotics SDKs. In contrast, the algorithm to look for faces in a camera image is much more complex. That is an example of a high-level function, and that is where the Cozmo SDK shines:

The most powerful feature of the Cozmo SDK is that it includes a comprehensive set of both low- and high-level functions, each one accessible with just a few lines of code. In the example above, turning backpack lights blue is one line of code; however, looking for faces in a camera image, is also just one line of code. That kind of feature set, combined with the physical capabilities of Cozmo, is what we believe makes the SDK so powerful.

Computer Vision

The camera is Cozmo’s most important sensor, providing a constant stream of camera images to the robot which are then analyzed. The Cozmo SDK allows you to access both the raw camera images as well as the entire built-in Vision System. That includes detecting, tracking, and recognizing faces, as well as estimating a head’s position and orientation in 3D. It also contains functions for recognizing Cozmo’s cubes via their markers and estimating their position and orientation in 3D space. Many other functions such as general marker detection, motion detection, cat and dog detection and tracking will be available soon after launch.

In this example, Cozmo gets upset when his cubes are not neatly aligned. We wait until the Vision System sees all three cubes. Then we check the orientation of each one. If any cube is out of alignment with the others, we play an upset animation. With just a few lines of code, we get access to the Vision System and the 3D pose estimation.

Localization and Path Planning

To interact with people and his environment in clever ways, Cozmo creates a map of his world containing all of the interesting things he sees. This includes cubes, people, his charger, and table edges. Cozmo uses the cubes not just as game elements, but also as landmarks to help him know where he is in the world. Every cube Cozmo sees gets added to the map as a landmark that can be used to figure out where he is. Combining the map with encoders and an IMU for measuring distance drive and orientation allows Cozmo to keep track of things happening around him and remember where they are even if he currently can’t see them.

In the first example below, we use Cozmo’s Path Planner to drive towards a goal even when obstacles are in the way. The last line, cozmo.PickupObject(), is the most important one in the example: based on the map created in the background, it launches the Path Planner to figure out what the robot needs to do to pick up that cube.

If Cozmo was right in front of the cube, this would be a simple straight path. In the example below, however, the Planner has to plan a path around the other cubes to reach the one selected. The code waits until Cozmo sees all 3 cubes, figures out which cube is furthest away, and then he drives there to pick up that cube.

The second video shows an example of the internal map Cozmo keeps at all times. The green zones are the areas Cozmo has already explored or is fairly confident that he can drive upon. This map is created automatically in the background at all times, and developers have access to all elements in it and can even add their own custom objects to it if needed.

Behaviors + Vision + Manipulation

Behaviors are systems in Cozmo where the AI takes control of the robot to achieve a goal. For example, the LookAround behavior enables Cozmo to look for interesting things in his world, such as faces and cubes. In the example below, after looking around for a few seconds, we check if the Vision System has located any cubes. If the robot finds zero cubes, the animation and sound system plays a short sad animation. If he finds one cube, another behavior called RollBlock runs, which plans a path to the cube and then uses the robot’s arm to roll the cube onto its side. If he finds two or more cubes, the StackBlocks behavior runs, which tells Cozmo to stack two given cubes on top of each other.

In addition to the Vision System and Path Planning, this example shows Cozmo’s manipulation system. Aligning precisely with cubes and then picking them up, rolling, or pushing them is a very complex task, involving many of the robot’s sensors and system. The Cozmo SDK provides access to such complicated functionality with very little code.

Sensors, Actuators, Display etc.

For developers interested in lower-level control of Cozmo, the SDK also provides direct access to low-level controls such as driving the robot’s treads, moving the head and lift, displaying bitmaps on his screen, reading accelerometer and gyroscope data, processing images from his camera, and communicating with the power cubes (lights, accelerometer, tap detection).

How does it work?

The first version of the SDK is written in Python (3.x). The SDK’s messaging layer is well documented, and anyone can port the SDK to pretty much any language they like. In order to use the SDK, here is what you will need:

1. A Cozmo robot

2.. A mobile device compatible with the Cozmo app (most iPhones, iPads, and newer Android phones or tablets), and

3. A computer with a USB port that can run Python 3.x. Most laptops, desktops, etc. with macOS, Windows or Linux should work.

The SDK is divided into two parts: the Cozmo Engine and the Cozmo SDK layer. The engine runs on a mobile device the same way the main Cozmo app does.
Cozmo communicates with the Engine over WiFi using a message protocol that provides access to several layers of robotics functions. This protocol is used during development and internally in the Cozmo App. The SDK is built to give developers an intuitive interface to this protocol. The mobile device becomes the interface to Cozmo; it connects to the robot via WiFi and to the computer via USB. As long as the computer can run Python 3.x, there is no need to install additional software. Later versions of the SDK will also allow developers to create native apps for iOS and Android similar to the Cozmo main app.

SDK Sign Up

Sign up to hear the latest info about the SDK available October 2016.

Get Cozmo 2 days early