LEGO Street View Car

May 2011

 



I’ve been working on this project for the guts of six months now, and you can probably see why!


It all started when was playing with the dGPS sensor from Dexter Industries, and realised that if I combined this sensor, with a robust chassis and a camera then I’d have a LEGO version of the famous Google Street View car. And what if I could then upload the pictures into Google Earth....It all seemed so easy... How wrong I was!



Current Status - Prototype!

This project is still very much in the proof-of-concept phase. It is a prototype with a lot of rough edges. The keyfob cameras are very temperamental and difficult to mount into standard LEGO dimensions.


Here are some quick facts and figures:


  1. Chassis is based on the Land Rover chassis made by Sariel. Pendular suspension, four wheel drive, rack-and-pinion steering on front wheels only.

  2. One NXT controls entire car.

  3. One dGPS from Dexter Industries to capture current coordinates.

  4. One PSPNx sensor from Mindsensors to receive commands from a PSP controller.

  5. One NXTBee sensor from Dexter Industries to transmit telemetry to a base station.

  6. One PFMate from Mindsensors to control the PF motors.

  7. Four 808 type 3 cameras ordered from China to capture the images around the car.

  8. One NXT motor for steering.

  9. One NXT motor to activate the shutter release mechanism on the cameras.

  10. Two PF XL motors to drive the front and rear wheels independently.

  11. One PF batter box, and one PF receiver.

  12. Control program written in NXC on the NXT using John Hansen’s enhanced 1.31 firmware, and NXC 1.2.1 r4 on Mac OS.


What works

  1. 1.The chassis can drive over grass carrying the full weight of all the components and batteries. That is one major achievement!

  2. 2.The dGPS works well when outdoors

  3. 3.I can record GPS waypoints in KML format.

  4. 4.The keyfob cameras are securely mounted

  5. 5.The NXTBee will send telemetry (battery level, GPS coordinates) to a receiver.


Yet to be done

  1. 1.Obviously there is no live streaming of video or pictures from the car to a receiver base-station. I have a design in mind to achieve this.

  2. 2.The keyfob cameras are very sensitive to incorrect placement. Aligning the tiny shutter button with a LEGO piece requires a lot of fine tuning. Not every camera fires concurrently. For now I can reliably capture images from the front-facing camera.

  3. 3.The process to copy the images from the car is manual and not yet automated.

  4. 4.I don’t have a way to stitch the pictures together into a panorama that you can explore in 3D. I can do a 2D panorama, but again it’s a manual process.

  5. 5.The car can’t drive itself - yet!


Design details

The LEGO Street View Car is controlled manually using a PSPNx sensor to receive commands from any standard PSP controller. The PSPNx sensor has the advantage of being simple to program against and supports both analog joystick input for steering and boolean button presses to activate the camera. Press the triangle button on the PSP controller to capture an image and log the GPS coordinates.


Every time an image is captured the current latitude and longitude are recorded from the dGPS. The NXT creates a KML format file in the flash filesystem which is then uploaded from the NXT to a PC. Opening the KML file in Google Earth shows the path that the car drove, and also has placemarks for every picture you took along the way. Click on the placemark to see the picture.


For each GPS coordinate I create a KML Placemark entry that embeds descriptive HTML code using the CDATA tag. The image link in the HTML refers to the last image captured on disk. The 808 camera starts with the filename EKEN0001.jpg for the first picture captured. The second picture captured is EKEN0002.jpg, and so forth.


<Placemark>

      <name>LSVC Snapshot 1</name>

      <description><![CDATA[<img src='DCIM/100MEDIA/EKEN0001.jpg' width=640 height=480> ]]></description>

<Point>

<coordinates> -6.185952, 53.446190, 0</coordinates>

</Point>

</Placemark>

    <Placemark>

      <name>LSVC Snapshot 2</name>

      <description><![CDATA[<img src='DCIM/100MEDIA/EKEN0002.jpg' width=640 height=480> ]]></description>

<Point>

<coordinates> -6.185952, 53.446190, 0</coordinates>

</Point>

</Placemark>


The snippet from the KML file gives you an idea of what each placemark should look like.


Once the car has finished driving press the orange button on the NXT to save the KML file. This writes a <pathstring> entry which records the actual path of the car. A path string is simply a list of coordinates that define a path in Google Earth along the Earth’s surface. For example:


<Placemark>

<name>LSVC Path</name>

<description>LSVC Path</description>

<styleUrl>#yellowLineGreenPoly</styleUrl>

<LineString>

<extrude>10</extrude>

<tessellate>10</tessellate>

<altitudeMode>clampToGround</altitudeMode>

<coordinates>

-6.185952, 53.446190, 0

-6.185952, 53.446180, 0

</coordinates>

</LineString>

</Placemark>


Is a path two coordinates not far from where I live.


From the NXT to Google Earth


How do we get the pictures and KML file from the NXT and into Google Earth? First of all we need to get all the data in one place. The KML file refers to the relative path of each image, so we can package the KML file and the images into a single directory.


An example of the output produced is shown below. In this test case I started indoors in my house and took a few pictures. As you can see the dGPS has trouble getting an accurate reading and so the pictures appear to be scattered around the map. I then drove the car outside and started to capture pictures as I drove. From Snapshot 10 onwards the images become more realistic based on where the car actually is.





You can see sample pictures captured by the car below:








The process flow to use the car is manual at the moment:


  1. 1.Attach the keyfob cameras to the PC using 4 USB cables

  2. 2.Delete all of the images on the cameras

  3. 3.Detach the cameras

  4. 4.Start the program on the NXT

  5. 5.Wait for the orange light to illuminate on the dGPS indicating a satellite fix is obtained.

  6. 6.The NXT display will show the current GPS coordinates.

  7. 7.Set the PSP controller to manual mode

  8. 8.Control the forward/reverse direction of driving with the left joystick, turn right/left with the right joystick.

  9. 9.Press the triangle button to take a picture and record the GPS coordinates

  10. 10.When done, press the orange button on the NXT.

  11. 11.Attach the NXT to the PC

  12. 12.Copy the Path.kml file from the NXT into an empty folder on the PC.

  13. 13.Attach each of the 4 USB cameras.

  14. 14.Take a group of 4 pictures across the cameras and merge into a panorama (STILL TO BE SOLVED).

  15. 15.Store the panorama into the folder containing the Path.kml

  16. 16.Open the Path.kml file in Google Earth and enjoy!



Source Code

How does it all work? I used NXC to program the LSVC, building on many available code libraries to control the third-party sensors. The code is pretty straightforward, after all the car does little more than start up and wait for commands from the PSP controller. One task is used to monitor the PSP controller and send commands to the motors, and the main task just runs a busy loop waiting for the orange button to be pressed to close the KML file.


The source files used are:

  1. 1.lsvc.nxc. This is the main source file that includes all of the other files used. It contains the constants for the sensors and motors, and the main loop. Also includes functions to communicate with the NXTBee to send/receive data.

  2. 2.logger.nxc. A helper set of functions for recording debug information with timestamps into a text file in the FLASH.

  3. 3.KMLCommon.nxc. Functions to create and write to the KML file.

  4. 4.DGPS-driver.nxc. Driver functions for the dGPS adapted from Xander’s original RobotC code.

  5. 5.PFMate-lib.nxc. The library from Mindsensors to control the PFMate sensor to send commands to the PF motors.

  6. 6.PSP-Nx-lib.nxc. The library from Mindsensors to read commands from the PSPNx sensor.


Simply copy all of these files into a directory and compile the lsvc.nxc file using the latest NXC compiler.


Video

Below you can see a video of the car in action at the Google Zeitgeist event in London. Martyn Boogarts, a fellow MCP4, was at the event and kindly shot the video as we drove around outside the Grove Hotel in London.

















Pictures

 

Last updated 8 June, 2011


All content © 2008 2009 2010 2011 Mark Crosbie  mark@mastincrosbie.com


LEGO® is a trademark of the LEGO Group of companies which does not sponsor, authorize or endorse this site. This site, its owner and contents are in no way affiliated with or endorsed by the LEGO Group. For more please read the LEGO Fair Play policy.