We have moved! Please visit us at ANTHROECOLOGY.ORG. This website is for archival purposes only.


Sep 30 2010

Autonomous EasyStar (Oh the Possibilities)

Although the Hexakopter has been performing flawlessly over the past few weeks, knock on wood; the Ecosynth team is still searching for a less expensive alternative for gathering aerial photographs. As of now, the only way to do this consistently and reliably is through the use of autonomous technology. Throughout this semester I have decided to convert the EasyStar platform into an inexpensive yet fully autonomous airplane.

One of the most significant advantages of this platform will be its ability to fly long distances without having any range limitations. This feature will enable us to fly an entire site without having to plan multiple flight patterns or plan around the “home” position of the aircraft. In addition to this, the open source code will enable us to adjust the flight characteristics and create subroutines of our own that will enhance the airplane’s usability and effectiveness. 

The primary objectives for this project are as follows:

  1. -  Create an autonomous airplane for under $400 (US)
  2. -  Capable of carrying a standard digital camera
  3. -  Utilize GPS technology to allow for multiple 3D waypoint navigation
  4. -  Maintain safe and stable autonomous flights in winds up to 10mph
  5. -  Provide continuous flight for approximately 15 minutes
  6. -  Relay the planes geographic position, altitude and airspeed to a field computer
  7. -  Return to home on request and sustain a holding pattern

 

If I somehow manage to complete all of the primary objectives with time to spare then I will begin to work on the slightly more interesting secondary objectives:

  • -  Ability to take off and land autonomously in large fields 
  • -  Autonomous parachute deployment at a specified position and altitude to assist with landing in small fields
  • -  Use IR sensors to provide obstacle avoidance capabilities
  • Over the past week I’ve been researching the various autopilot systems available online. One of the most promising options is the ArduPilot from DIY Drones. The main advantage of this system is its use of open source code which can be modified to accommodate any airplane with three or more channels. The ArduPilot is also capable of controlling additional channels to support optional tasks, such as parachute deployment. In addition, the ArduPilot is capable of performing 3D waypoint navigation and two-way telemetry data transfer via the optional XBee modules. DIY Drones provides a free point and click style mission planner that enables users to easily create and upload 3D waypoints to the airplane (as illustrated in the figure above). The components required to use the ArduPilot include the following: 
  • ArduPilot Board                                24.95
  • Shield V2 kit w/ Airspeed Sensor       57.20
  • Breakaway header                              2.50
  • U-Blox5 GPS                                   87.90
  • U-Blox Adapter                                19.50
  • U-Blox Cable                                   1.95
  • X, Y and Z Sensors                           99.90
  • Female-to-female RC cables (x4)       1.50 per
  •  
  • The parts required for wireless telemetry are as follows:
  • Air XBee                                            42.95
  • Ground XBee                                      44.95
  • XBee Antenna                                  7.95
  • Adafruit adapter board                      10.00
  • XBee Explorer USB                          24.95

The total cost of the ArduPilot system, without wireless telemetry; is around $299.90 (US). The wireless telemetry brings the total cost to $430.70. Throughout the upcoming week I will continue to look for ways of reducing the cost of this system. If we decide to use the ground XBee, XBee Explorer USB and XBee Antenna included with the Hexakopter then the total cost could be reduced to $352.85, which is still a significant investment.

The EasyStar itself is a relatively inexpensive plane; an almost ready to fly (ARF) version can be purchased for around $120. Our prior experience with the EasyStar has led us to believe that the stock motor and ESC should be upgraded to a 400 speed brushless in-runner motor and brushless 25A ESC. In addition to this, the tail rudder should be extended by 1.5 in. to allow for a tighter turning radius. Over the next few weeks I’ll be creating a page which will contain all of the modifications made to our EasyStar as well as their overall effectiveness.

You can download the Arduino software here and the waypoint configuration tool here, both of which are free. The latest version of the autopilot code can be found here. I will be sure to post more blogs about the ArduPilot system as I continue to familiarize myself with its tools and capabilities.     


Sep 25 2010

Camera Exposure Calibration

Even though we have had great success with the Canon and Casio continuous shooting**, high speed cameras for getting high image overlap, we are still having issues with image exposure.  I purchased a Lastolite EzyBalance camera calibration card from Service Photo in Baltimore as way to systematically deal with these issues.

When in continuous shooting mode, the cameras make calculations for focus and exposure based on the first photo taken when the button is pressed (Canon SD4000 Camera Manual).  This means that all photos in the scene will have an exposure (under-, over-, or “correct”) based on the lighting conditions of the first photo.  We discovered this when first using these cameras on the Slow Sticks.  When attaching the camera to the underside of the plane frame, the camera is pointed up at the sky and sun.  When the continuous shooting mode is activated the camera records the light conditions as if it were receiving direct light from the sky/sun.  When the plane is turned right side-up less light is entering the lens and so the photos are underexposed, too dark.  If I mount the camera and activate continuous shooting mode with the camera pointed at the ground it will record lower lighting conditions than what will be observed at altitude above the canopy, so the photos are overexposed, too light.

I am still learning about how SIFT and computer vision work and we are just now at the point where we can start to test changes in camera settings, but based on some preliminary research I think it will be important to strive for consistent illumination among images.  SIFT is largely invariant to changes in illumination between images, so it should still be possible to match photos of the same place under slightly varying illumination conditions (Lowe 1999).***  Since the camera settings are consistent between photos, there should not be changes in feature illumination between photos for the same image collection, unless clouds move into the scene during the flight.  However, under- or over- exposed images may result in a reduction in the detection of image features.  Many things to be tested for sure, but I want to start with trying to achieve consistent image illumination.

Here are some simple examples of my backyard for illustration.  I used the open-source GIMP image editor to generate the image intensity histograms for each image.  The interpretation of image intensity histograms is somewhat subjective or scene based, but the examples below merely serve as illustration of the value of the cal panel. 

The image on the left is over-exposed and the image on the right is under-exposed.  Prior to setting the camera into continuous shooting mode I pointed the lens down at the shadow of my body at my feet and then out across my lawn, resulting in the over-exposed image at left.  For the right image I started with the camera pointed up at the sun and then down to my lawn, resulting in under-exposure.  The left histogram is so white that almost all values are at the far right end of the chart and are hard to see.  The right histogram has values clumped at the left side, representing darker values throughout the whole scene.  In either image it is difficult to make out features, for example the grass in shadow at the right side of the right image, and of course nothing is visible in the left image.

By photographing a fully illuminated grey calibration panel first I get a resulting image with much more natural looking and distributed color intensity, as can be seen in the image at right.  This more spread out histogram is interpreted as having more tonal variation.  While we still have lots to test about camera settings, the goal is that by using this cal panel prior to flight we will be able to achieve consistent photo illumination and exposure.  There are other panels with black, grey and white that can be used to deliberately cause images to be under or over exposed, e.g., the Lastolite XpoBalanceused by some to calibrate digital photos for portraits and also for calibrating the intensity of LiDAR beams (Vain et al. 2009).

OK, that’s enough.  It is too beautiful out to drag this post along any more!

 

** I just discovered a ‘low-light’ 2.5MP resolution camera setting that makes it possible to achieve 5 photos per second with the Canon SD4000, wow!  This has the effect of increasing the camera ISO which may result in grainy photos under high illumination and it is not possible to change that resolution setting.

*** Thanks to my Computational Photography course that I am taking this semester, my review of the Lowe SIFT paper for this post finally made sense! 

refs:

Lowe, D.G. 1999. Object recognition from local scale-invariant features. In International Conference
on Computer Vision
, Corfu, Greece, pp. 1150-1157.

Vain A., Kaasalainen S., Pyysalo U., Krooks A., Litkey P. Use of Naturally Available Reference Targets to Calibrate Airborne Laser Scanning Intensity Data. Sensors. 2009; 9(4):2780-2796.

Sep 23 2010

First Sucessful Multi-waypoint Hexakopter Flight! (Video)

Last Friday we had our first sucessful multi-waypoint Hexakopter flight. Here is a video of that amazing achievement.

Sep 21 2010

You've got to let the little guy's leash out!

This afternoon I worked with Nisarg to better understand how to setup a path for collecting aerial photos with the Hexakopter.  We quickly found that doing so would not be difficult and that adding our own custom waypoint list to the Mikrotool was straightforward.  However, during the fligh test we learned about an important setting to be aware of in the Navi-Ctl that really limited our test today.  By changing the settings as described below, we should be out flying over the entire Knoll in no time!  I think Nisarg is going to post later about the Mikrotool Map software that he was working with that provides an easy system for getting a Google Earth image into the Mikrotool OSD.

 

In testing our waypoint options, Nisarg was able to create an output text file (a .wpl file) of the Mikrotool OSD waypoints.  The file looks like this:

[General]
FileVersion=1
NumberOfWaypoints=10
[Waypoint0]
Latitude=39.253988
Longitude=-76.71177
Radius=10
DelayTime=1
[Waypoint1]
Latitude=39.251826
Longitude=-76.711718
Radius=10
DelayTime=1
[Waypoint2]
...
...

I looked at that and thought, “Well I can make that!” and set about generating a waypoint file over a route we wanted to fly for the Knoll site on campus in ArcGIS.  We had been discussing flight strategies and how to achieve image overlap and I showed Nisarg and Garrett a little bit of the kind of map making and simple spatial functions that can be done in ArcGIS: Garrett seemed more impressed than Nisarg.  I have been consulting Elements of Photogrammetry a lot recently for flight planning and also some simple calculations I made of the amount of overlap and area observed by a camera at a given height above the ground surface. I will be posting more about that later.

Anyway.  We came up with the a simple parallel grid track for the Hexakopter to navigate over the Knoll with 50m spacing between tracks.  I estimated that at about 70m altitude from the ground (@ Home) I would achieve the desired overlap even over the Knoll and trees. We went outside with the gear, climbed up to the top of the parking garage and went for it.  We also had a Garmin Edge GPS attached to the top of the dome to provide a track of where the little guy was going: not sure if the MKGPS can record tracks.  From the beginning it seemed like something was wrong.  It was kind of flying in the correct pattern, but it did not seem like it was going far enough across the forest.  We brought it back down and took a look at the GPS track.

As you can see in this somewhat messy image, the little guy did not go exactly where we wanted him to. 

A brief legend:

  • - Blue lines are the intended route
  • - Green dots are the intended waypoints given to the Mikrokopter
  • - House symbol is flight home
  • - Large yellow arc is a 250m buffer around the quad, more on this below
  • - Small yellow circle is a 100m buffer around fight home

So what happened?  The Hexakopter tried its best to fly those waypoints, but kept hitting an invisible wall and was forced to come back.  The units come preset with a GPS maximum range of 100m.  So our guy was flying his path, hit that 100m mark and was deflected along his route without hitting all the points.  Through a bit of research we came across the issue on the forums, http://forum.mikrokopter.de/topic-13563.html, and the solution on the Wiki (linked from the forums): http://www.mikrokopter.de/ucwiki/MK-Parameter/Navi-Ctrl_2 .

The units still will not fly beyond 250m with GPS with the current setup.  We thought that this was the default setting and hence set up our route to fly at max 250m across the Knoll from an intended Home in the quad, that is why the larger yellow arc is not centered. 

From here we are going to update the Navi-Ctl setting to allow us to fly to the max 250m from home and I am going to put together a small script in Python to turn an ArcGIS point file table into the waypoint format needed for the Mikrotool…but maybe not by Friday!

refs: Wolf, P., DeWitt, B., 2000. Elements of Photogrammetry with Applications in GIS 3rd Edition, McGraw Hill.

Sep 20 2010

The Mikrokopter Lives!

My, how far we’ve come! 

Just about one year ago I was out flying my kite almost everyday to get coverage over our two study sites on the UMBC campus.  Over this past week we have made a huge step forward, a systematic ‘test’ flight with the Mikrokopter Hexacopter over the Herbert Run forests.

Flying the large delta conyne kites (like the one shown here, image credit Into The Wind Kites http://www.intothewind.com/) was fun and got the camera in the air, but it was very hard to control both the altitude of the camera and its position over the forest.  This meant it was very difficult to test flight plans, or even begin to get at understanding the best flight plan strategy for use with computer vision.

Over the past summer we worked with several students from the UMBC GES and MECHE departments and a visiting intern from Clark University (thanks Evan, Nisarg, Garrett, and Noam) with the goal of using hobbyist aircraft to carry the cameras.  We moved away from using the Canon CHDK camera setup, instead using high-speed (~2 photos / sec) cameras with continuous shooting modes to collect huge numbers of overlapping photos.  We had a lot of promising flights and successes with the hobbyist aircraft, the Slow Sticks and Easy Stars.  But we also had a lot of technical challenges and crashes that made us question the sustainability and repeatability of the ultra-cheap systems for our scientific research and technological development stage.

Enter the Mikrokopter Hexacopter.  The Mikrokopter line of remote controlled aircraft offers precision control and GPS navigation.  Last Friday we made our first demonstration of the GPS-assisted navigation over the Herbert Run site.  The Photosynth generated from those photos is here, http://bit.ly/bqAhzL, and while it looks similar to the rest of our aerial synths, it is generated with photos taken along a pre-designated path at a constant altitude.  Remarkable!

I expect things to progress quickly this fall (that dissertation is calling) and we have set up another blog for weekly progress about the nitty-gritty of Ecosynth research, http://ecotope.org/ecosynth/blog/.  I will continue working with this blog as a reference for the methods and research progress and the ‘weekly’ should be a place to go for latest in weekly goings-on in the Ecosynth lab.

Thanks team, we could not have gotten here without all of your hard work.

Sep 17 2010

We get a new Tx/Rx and the Hexakopter flies waypoints!

Last week we had ordered a DX7 2.4 GHz Spektrum and an AR7000 DuaLink 7 channel receiver transmitter/receiver set from an online company. Unfortunately they had to send back their entire stock of Spektrum Tx/Rx due to a binding issue. On Monday Jonathan went down to Hobby Town USA and bought the set so that we could work on it during the week. By Tuesday morning, Jonathan had the transmitter charged so that we could work on binding it to the Hexakopter. First we had to pair the transmitter with the receiver, which we accomplished by powering the Rx with a small 5V NiCaD and pressing the bind button on the back of the Tx. Once the LED on the Rx glowed orange we tested it by hooking up a servo and moving the appropriate stick on the Tx.

Surprisingly, there was little work to do in order to connect the new receiver to the Flight Control board. We separated the main receiver from the external receiver. Then we clipped off one end so that we could solder the wires to the Flight Control board. Using a picture from the Mikrokopter Wiki we were able to solder the connections without a problem. Once we finished wiring the receiver we turned on the Hexakopter and the transmitter. However we soon found out via the “alarm clock style” buzzing from the Hexakopter that it was not receiving any signal. After a few minutes of trying to figure out what could be the issue, I changed the receiver setting on the Mikrokopter Tool from multi-signal (PPM) to spektrum satellite. This fixed the connection issue and allowed us to set up the altitude and GPS control.

Thursday we decided to test the Hexakopter and the new transmitter. First we tested the altitude control.  It was set up through the Mikrokopter Tool to be an adjustable altitude lock. We set this up so that once the altitude lock was set, you could adjust the height of the Hexakopter by moving the throttle stick up or down - a neutral stick meant that the Hexakopter was to hold position. After a few tries we managed to get the Hexakopter into altitude lock without a problem. We did notice however, that when the altitude lock was engaged near ground level (around 8 meters) the Hexakopter started to oscillate between around the locked height. We believe that this was caused by the propeller backwash and decided that any further flights would not engage the altitude control near the ground for extended periods. Afterwards we attempted to test the system's waypoint software. We started with the GPS lock mode, which held the Hexa at a position determined by when a 3-way switch on the Tx was moved to the middle position. The GPS lock mode had little issues and was easy to engage. However the main issue came when attempting to set waypoints. We could get it to fly to a single position by specifying a waypoint and switching the 3-way switch to GPS home mode but we were unable to get it to fly multiple waypoints.

Finally after some research on the Mikrokopter.us forums, we were ready to fly a set of waypoints. The issue was that we were not setting altitude lock and GPS lock before attempting to load. On Friday we set out to Herbert Run to test if the Hexa could fly a set of waypoints.