We have moved! Please visit us at ANTHROECOLOGY.ORG. This website is for archival purposes only.


Oct 22 2010

Weekly Flight Progress (Week of 10/17/10)

This past week I got in 3 good flights at UMBC and New Jersey and learned a lot more about how to get things working well.

The first flight was at the pine barrens site in Pemberton, NJ.  I arrived at the site on Saturday to scout things out.  I met the local forest researcher Ken Clark and he showed me around the plot where I would be flying and we checked out the forest from atop a modular tower used for making meteorological measurements; I was not crazy about climbing the tower, but Ken had no problem.  I was up at 6am Sunday and at the site by 7:30.  By about 8am I had inflated all of my new huge 3' diameter balloons and set out to place these new aerial markers in the field.  This ended up working pretty well, although I think I put a tiny hole in one balloon and several balloons popped between the time I set them up and when I went to retrieve them.  It was gusting to about 13mph (according to the local met data, thanks Ken!) and the balloons blew around and into the trees throughout the morning.  Overall the flight went well, but I think that because of the gusty winds, the Hexa got blown of course several times and we may have issues with image overlap during the 3D reconstruction. Site prep took about 3 hours as it was more challenging to place the balloons in the field than when I am just using buckets on the ground.

The next flights were both on Thursday at the UMBC Herbert Run location.  I had everything set up by about 11am and then commenced with flight testing.  I think it is a good idea to use a spare battery for testing out the flying conditions on site prior to the actual image collection mission.  I found that the GPS and altitude were holding well, but that it was still quite windy.  At this site I use buckets instead of balloons as they are more stable and there are more open areas to place the buckets.  Things went well on the first flight, but the Xbee inexplicably cut out quite often.  I thought it was odd that the Xbee cut out so frequently because there was almost no data connection problem at the New Jersey site earlier in the week.  Also, it was apparent from the Garmin Astro track that the route was being affected by the wind.  I landed the unit after the flight and swapped in a new lipo, new camera battery and new camera memory card and proceeded with another flight at an altitude 40m above the first flight and along the same route.  The track of this flight showed a similar pattern as the previous one.  I posted more about the flight paths and looking at the tracks in an earlier post this week, here.  In all I got in two full flights in under an hour, by myself, with about one hour site prep. 

 

In summary, I had three good flights this week that I setup and flew on my own.  It looks like at each flight winds above 13 mph may have caused the Hexa to deviate from the flight path, hopefully not to the detriment of image overlap.  I think the procedure for preparing gear and setting up the site is finally getting honed down to a smooth operation.  And thank goodness for my station wagon!

Oct 19 2010

3D and Spectral Remote Sensing with Computer Vision

Wow, what amazing progress!  I posted a few weeks ago when we were just starting to get the Hexakopters working, how excited I was when I considered that we were still flying kites just ONE YEAR AGO!  Now it is becoming a reality that the Mikrokopters can really move this interdisciplinary research fusion of ecological remote sensing and computer vision into a reliable system for making 3D, spectral color measurements of ecosystem vegetation for measuring biomass and species diversity.  

There is definitely a lot to learn about the process, but we have got the flying down pretty well for data collection.  The video here is of me flying the Hexa up through a large gap in the canopy at the Knoll site at UMBC.  This is an invaluable capability of this system (and its pilot!) that makes it possible to fly sites like the Knoll and SERC, where it is not possible to be centrally located in a large open clearing.  

DRAFT: comparison at HR sites, seasons

Oct 19 2010

Flight Telemetry Data and Monitoring the Flight

We recently learned about the robust telemetry data that the Flight Control board creates when a MicroSD card is plugged into the onboard slot.  Since then we have always flown with the card in to capture the telemetry data and it is proving to be an invaluable part of the post-flight assessment process.

The next steps will require a *simple* GIS workflow to evaluate whether the actually flown path sufficiently matches the pre-planned path to provide the necessary image sidelap and endlap for 3D reconstruction.  

We have already seen first hand how deviations in the fight plan tracks result in gaps in the vision reconstruction and also how mis-aligned platform orientation results in gaps as well.  At a flight at SERC the platform and camera where oriented 90 degrees from the path of travel, resulting in no sidelap between parallel tracks, link to Photosynth.  At a recent flight in New Jersey the wind blew the little guy around a lot and even though the camera orientation was spot on, the width between some parallel tracks exceeded our flight plan for achieving side lap, again resulting in large scan gaps, link to Photosynth.

So what does this mean?  It is necessary not only to have spot-on preparation of camera orientation and flight planning prior to flight, but it will be necessary to run diagnostics in the field to evaluate whether the route flown was sufficient for 3D reconstruction.  I wrote a simple python script to translate the Mikrokopter GPX telemetry data into a text file for use in a spreadsheet or GIS program.  By doing this we can look at characteristics of the flight system through time (e.g., battery voltage vs. flight time) or space (e.g., Navi-Ctl status messages at each GPS point along the flight path).  The Python script that translates Mikrokopter GPX telemetry data to a text file is located on our Coding Corner page.

Oct 19 2010

Pre-Flight Camera Settings & Calibration

DRAFT: Recall a few weeks ago I posted about the camera exposure calibration card I purchased from Service Photo.  The call card is in full use as part of the pre-flight set up, but after several good and bad field experiences it is necessary to revise my initial procedure.

Oct 19 2010

The "Ups" and "Downs" of Remote Sensing with the Mikrokopter

DRAFT: We have had great successes with the Mikrokopter Hexakopter for collecting digital photos for 3D computer vision reconstruction, but there are still many challenges yet to be overcome. 

We are now able to plan a flight with GIS to cover a particular 250m x 250m area with a desired amount of image overlap and translate that into a flight plan for the Mikrokopter to follow.  We have successfully demonstrated this at each site, however some other complications have become apparent. 

One challenge that remains is the control of camera exposure.  I purchased a camera calibration card a few weeks ago and have incorporated that into the pre-flight checklist, but that technique is not quite perfected.   

Oct 07 2010

Back in the air and doing great!

One week after a nasty crash at one of our suburban forest study sites, we are back in the air and *hopefully* back on track for a great collection of vegetation dynamics this month.

More on the crash and the rebuild on the Weekly are forthcoming, the team has mid-terms right now, but I wanted to post about the recovery and success today.  We had two Hexas on order and they arrived on Thursday of last week.  Nisarg and Garrett put in the time this past weekend to get the two new birds up and running and we had a few great tests in a parking garage and in an empty gymnasium, inside because of a cold autumn rain.

On Monday we calibrated the receivers to the units and everything appeared ready for great flights once the weather broke.  The forecast called for rain until Thursday and then becoming beautiful.  The forecast for today included rain but the morning and midday had light winds and mostly overcast skies so we decided to go for it.  I knew the gear was ready and I pulled out some of my new pre-flight checklists to prepare the field kit.  Garrett showed up and helped with the packing while I prepped the official flight path in ArcGIS.

Here is the breakdown of field work by time:

  • 10:30 – Arrive at site and unload (leave note on car begging not to be ticketed)
  • 11:30 – All markers are placed with position recorded in GPS
  • 12:00 – Back to site and ready to go after realizing I left camera battery in lab
  • 12:05 – Garrett has arrived and we are powering up Hexa
  • 12:30 – Updated and verified Mikrotool settings, unit has GPS lock, ready to go…
  • 12:33 – Take off!
  • 12:48 – Jonathan grabs Hexa as it makes its descent. Touchdown.
  • 13:05 – Gear is packed up; Depart site
  • 13:30 – All gear has been unloaded and we are back in lab downloading data

In sum, about 2 hours on-site for what appears to be a perfect 15 minute data acquisition.  I expect that we can take off at least one hour from this as we get better at making acquisitions.  A Photosynth of the photos is running now and I will update with a link when it is ready.  I have also attached below the KML file of the track downloaded from the MicroSD card on board the Hexa’s navigation control board.

HR_20101006_Mikrokopter_GPS_telemetry.KML (49.45 kb)

UPDATE: The Photosynth run is finished, http://photosynth.net/view.aspx?cid=011796ed-ed9e-43d5-bc0a-f5a80bcae7d6 . This was based off of every other photo in the set, amazing.  I can't wait to get some seasonal change in the canopy, it will look beautiful.

 

Sep 25 2010

Camera Exposure Calibration

Even though we have had great success with the Canon and Casio continuous shooting**, high speed cameras for getting high image overlap, we are still having issues with image exposure.  I purchased a Lastolite EzyBalance camera calibration card from Service Photo in Baltimore as way to systematically deal with these issues.

When in continuous shooting mode, the cameras make calculations for focus and exposure based on the first photo taken when the button is pressed (Canon SD4000 Camera Manual).  This means that all photos in the scene will have an exposure (under-, over-, or “correct”) based on the lighting conditions of the first photo.  We discovered this when first using these cameras on the Slow Sticks.  When attaching the camera to the underside of the plane frame, the camera is pointed up at the sky and sun.  When the continuous shooting mode is activated the camera records the light conditions as if it were receiving direct light from the sky/sun.  When the plane is turned right side-up less light is entering the lens and so the photos are underexposed, too dark.  If I mount the camera and activate continuous shooting mode with the camera pointed at the ground it will record lower lighting conditions than what will be observed at altitude above the canopy, so the photos are overexposed, too light.

I am still learning about how SIFT and computer vision work and we are just now at the point where we can start to test changes in camera settings, but based on some preliminary research I think it will be important to strive for consistent illumination among images.  SIFT is largely invariant to changes in illumination between images, so it should still be possible to match photos of the same place under slightly varying illumination conditions (Lowe 1999).***  Since the camera settings are consistent between photos, there should not be changes in feature illumination between photos for the same image collection, unless clouds move into the scene during the flight.  However, under- or over- exposed images may result in a reduction in the detection of image features.  Many things to be tested for sure, but I want to start with trying to achieve consistent image illumination.

Here are some simple examples of my backyard for illustration.  I used the open-source GIMP image editor to generate the image intensity histograms for each image.  The interpretation of image intensity histograms is somewhat subjective or scene based, but the examples below merely serve as illustration of the value of the cal panel. 

The image on the left is over-exposed and the image on the right is under-exposed.  Prior to setting the camera into continuous shooting mode I pointed the lens down at the shadow of my body at my feet and then out across my lawn, resulting in the over-exposed image at left.  For the right image I started with the camera pointed up at the sun and then down to my lawn, resulting in under-exposure.  The left histogram is so white that almost all values are at the far right end of the chart and are hard to see.  The right histogram has values clumped at the left side, representing darker values throughout the whole scene.  In either image it is difficult to make out features, for example the grass in shadow at the right side of the right image, and of course nothing is visible in the left image.

By photographing a fully illuminated grey calibration panel first I get a resulting image with much more natural looking and distributed color intensity, as can be seen in the image at right.  This more spread out histogram is interpreted as having more tonal variation.  While we still have lots to test about camera settings, the goal is that by using this cal panel prior to flight we will be able to achieve consistent photo illumination and exposure.  There are other panels with black, grey and white that can be used to deliberately cause images to be under or over exposed, e.g., the Lastolite XpoBalanceused by some to calibrate digital photos for portraits and also for calibrating the intensity of LiDAR beams (Vain et al. 2009).

OK, that’s enough.  It is too beautiful out to drag this post along any more!

 

** I just discovered a ‘low-light’ 2.5MP resolution camera setting that makes it possible to achieve 5 photos per second with the Canon SD4000, wow!  This has the effect of increasing the camera ISO which may result in grainy photos under high illumination and it is not possible to change that resolution setting.

*** Thanks to my Computational Photography course that I am taking this semester, my review of the Lowe SIFT paper for this post finally made sense! 

refs:

Lowe, D.G. 1999. Object recognition from local scale-invariant features. In International Conference
on Computer Vision
, Corfu, Greece, pp. 1150-1157.

Vain A., Kaasalainen S., Pyysalo U., Krooks A., Litkey P. Use of Naturally Available Reference Targets to Calibrate Airborne Laser Scanning Intensity Data. Sensors. 2009; 9(4):2780-2796.

Sep 21 2010

You've got to let the little guy's leash out!

This afternoon I worked with Nisarg to better understand how to setup a path for collecting aerial photos with the Hexakopter.  We quickly found that doing so would not be difficult and that adding our own custom waypoint list to the Mikrotool was straightforward.  However, during the fligh test we learned about an important setting to be aware of in the Navi-Ctl that really limited our test today.  By changing the settings as described below, we should be out flying over the entire Knoll in no time!  I think Nisarg is going to post later about the Mikrotool Map software that he was working with that provides an easy system for getting a Google Earth image into the Mikrotool OSD.

 

In testing our waypoint options, Nisarg was able to create an output text file (a .wpl file) of the Mikrotool OSD waypoints.  The file looks like this:

[General]
FileVersion=1
NumberOfWaypoints=10
[Waypoint0]
Latitude=39.253988
Longitude=-76.71177
Radius=10
DelayTime=1
[Waypoint1]
Latitude=39.251826
Longitude=-76.711718
Radius=10
DelayTime=1
[Waypoint2]
...
...

I looked at that and thought, “Well I can make that!” and set about generating a waypoint file over a route we wanted to fly for the Knoll site on campus in ArcGIS.  We had been discussing flight strategies and how to achieve image overlap and I showed Nisarg and Garrett a little bit of the kind of map making and simple spatial functions that can be done in ArcGIS: Garrett seemed more impressed than Nisarg.  I have been consulting Elements of Photogrammetry a lot recently for flight planning and also some simple calculations I made of the amount of overlap and area observed by a camera at a given height above the ground surface. I will be posting more about that later.

Anyway.  We came up with the a simple parallel grid track for the Hexakopter to navigate over the Knoll with 50m spacing between tracks.  I estimated that at about 70m altitude from the ground (@ Home) I would achieve the desired overlap even over the Knoll and trees. We went outside with the gear, climbed up to the top of the parking garage and went for it.  We also had a Garmin Edge GPS attached to the top of the dome to provide a track of where the little guy was going: not sure if the MKGPS can record tracks.  From the beginning it seemed like something was wrong.  It was kind of flying in the correct pattern, but it did not seem like it was going far enough across the forest.  We brought it back down and took a look at the GPS track.

As you can see in this somewhat messy image, the little guy did not go exactly where we wanted him to. 

A brief legend:

  • - Blue lines are the intended route
  • - Green dots are the intended waypoints given to the Mikrokopter
  • - House symbol is flight home
  • - Large yellow arc is a 250m buffer around the quad, more on this below
  • - Small yellow circle is a 100m buffer around fight home

So what happened?  The Hexakopter tried its best to fly those waypoints, but kept hitting an invisible wall and was forced to come back.  The units come preset with a GPS maximum range of 100m.  So our guy was flying his path, hit that 100m mark and was deflected along his route without hitting all the points.  Through a bit of research we came across the issue on the forums, http://forum.mikrokopter.de/topic-13563.html, and the solution on the Wiki (linked from the forums): http://www.mikrokopter.de/ucwiki/MK-Parameter/Navi-Ctrl_2 .

The units still will not fly beyond 250m with GPS with the current setup.  We thought that this was the default setting and hence set up our route to fly at max 250m across the Knoll from an intended Home in the quad, that is why the larger yellow arc is not centered. 

From here we are going to update the Navi-Ctl setting to allow us to fly to the max 250m from home and I am going to put together a small script in Python to turn an ArcGIS point file table into the waypoint format needed for the Mikrotool…but maybe not by Friday!

refs: Wolf, P., DeWitt, B., 2000. Elements of Photogrammetry with Applications in GIS 3rd Edition, McGraw Hill.

Sep 20 2010

The Mikrokopter Lives!

My, how far we’ve come! 

Just about one year ago I was out flying my kite almost everyday to get coverage over our two study sites on the UMBC campus.  Over this past week we have made a huge step forward, a systematic ‘test’ flight with the Mikrokopter Hexacopter over the Herbert Run forests.

Flying the large delta conyne kites (like the one shown here, image credit Into The Wind Kites http://www.intothewind.com/) was fun and got the camera in the air, but it was very hard to control both the altitude of the camera and its position over the forest.  This meant it was very difficult to test flight plans, or even begin to get at understanding the best flight plan strategy for use with computer vision.

Over the past summer we worked with several students from the UMBC GES and MECHE departments and a visiting intern from Clark University (thanks Evan, Nisarg, Garrett, and Noam) with the goal of using hobbyist aircraft to carry the cameras.  We moved away from using the Canon CHDK camera setup, instead using high-speed (~2 photos / sec) cameras with continuous shooting modes to collect huge numbers of overlapping photos.  We had a lot of promising flights and successes with the hobbyist aircraft, the Slow Sticks and Easy Stars.  But we also had a lot of technical challenges and crashes that made us question the sustainability and repeatability of the ultra-cheap systems for our scientific research and technological development stage.

Enter the Mikrokopter Hexacopter.  The Mikrokopter line of remote controlled aircraft offers precision control and GPS navigation.  Last Friday we made our first demonstration of the GPS-assisted navigation over the Herbert Run site.  The Photosynth generated from those photos is here, http://bit.ly/bqAhzL, and while it looks similar to the rest of our aerial synths, it is generated with photos taken along a pre-designated path at a constant altitude.  Remarkable!

I expect things to progress quickly this fall (that dissertation is calling) and we have set up another blog for weekly progress about the nitty-gritty of Ecosynth research, http://ecotope.org/ecosynth/blog/.  I will continue working with this blog as a reference for the methods and research progress and the ‘weekly’ should be a place to go for latest in weekly goings-on in the Ecosynth lab.

Thanks team, we could not have gotten here without all of your hard work.

Aug 18 2010

3D Scanning of UMBC?

 

Ready for an Ecosynth scan over the entire UMBC campus?  I think it’s time!

I just gave a brief talk about our 3D mapping work to the University administration at the annual retreat.  Along with Stu Schwartz (Senior Scientist, CUERE), Suzanne Braunschweig (Lecturer, UMBC GES), and Patricia La Noue (Director, UMBC Dept. of Interdisciplinary Studies) I was on a panel discussing the value of UMBC’s natural spaces as classroom and laboratory.  I spent my 5 minutes talking about how I use the forests on campus as my lab for developing our new approach to ecological remote sensing.  Suzanne talked about her experiences teaching science classes using the natural environment of UMBC, Stu talked about the campus as a lab for studying the hydrology and planning side of stormwater management strategies, and Patricia talked about her work engaging students of interdisciplinary studies with UMBC’s natural spaces through the Greenway project.

I ended my talk with this point cloud image from the Herbert Run site that really captured the 3D structure of buildings and trees around the dorms of campus, I think it was a big hit!  Link to the Photosynth, here.  I mentioned in this slide how we are thinking about an Ecosynth scan of the whole campus.  Afterwards several people came up to ask about Ecosynth and about a campus ecological inventory.

The area inside the loop is about 63 hectares, easily 10 times bigger than anything we have done before.   But, I think it is possible.  We met with another RC flier on Monday who is a member of the Baltimore Area Soaring Society and is very excited by the value that our work places on his hobby.  He thinks that the Slow Stick might be a great aerial platform for a campus acquisition simply because it requires minimal space for take off and landing (recall that flight from 7/30 where we staged from atop a parking garage).  So, I will have to see how a 3D scan of campus fits in with my schedule of dissertation work.  I think I will need to get some help!  My slides are attached in PDF form below.

 

DANDOIS_ELLIS_UMBC_Ecosynth_short.pdf (3.94 mb)