Because the Ecosynth team has been looking at purchasing the GoPro Hero2 as our next platform camera I figured I'd share some of the videos I've taken with the GoPro Hero. These videos were recorded over the past summer in Cockeysville Maryland. The body of water that's visible when flying at higher altitudes is loch raven reservoir. The GoPro hero is one of the lightest and least expensive HD Cameras on the market and I'm very pleased with its performance so far. The videos were uploaded in HD remember to turn HD on.

http://www.youtube.com/watch?v=Qz9Sl9JgQRg

http://www.youtube.com/watch?v=J3gTYgJlDuw&feature=related

I'm thinking about trying to use statistics to pick a good range of colors to search through the point clouds to pick out an object of a certain color. Right now I'm just looking for the bright orange buckets that were placed throughout the test site. I searched online and picked out all of the RGB values for colors I thought were close enough to orange and found their mean and standard deviation. For orange I got a mean RGB of (245, 110, 20), which corresponds to this color here. Maybe it would be a good idea to allow the user to pick how many standard deviations to go out to (1, 2 or 3) in case some points are missed.

ArcMap was also provides these statistics given a set of selected points, and I used this data to test run the first part of the code I wrote. I picked what I thought was a suitable range of RGB values and ran the code for a point cloud overlayed on an image of UMBC. This image also included the locations of all of the orange buckets that were placed and captured in the scan. The code generated a set of points that fell in RGB range, and when plotted on this image, it seemed to work for most of the locations. The main problem with this, is the code found a lot of points where there was obviously not a bucket, but something larger like a car or the huge building on campus that has a slightly orange roof.

The original idea was only to indicate one (x,y,z) coordinate for every clump of orange points within a certain volume, which would indicate one bucket. Maybe a way to remedy the extra points, besides tuning the color range, would be to limit to how far out a "density" of these points should be maintained. For example, if a density of 10 points per cubic meter was maintained for over 1 cubic meter, that area may indicate something larger than a bucket. This means I need to find an appropriate density and volume that would indicate a bucket in the point cloud, and get rid of anything that has this density for a volume larger than the estimated bucket's volume.

“Breakthrough technology enables 3D mapping of rainforests, tree by tree” - the latest news from the Carnegie Airborne Observatory (CAO)- but also old news: since about 2006, the CAO has been the most powerful 3D forest scanning system ever devised, and Greg Asner has continually improved it.

The CAO was the original inspiration behind Ecosynth.  In 2006/2007, I  was on sabbatical at the Department of Global Ecology at the Carnegie Institute of Washington at Stanford, and my office was right next to Greg’s.   Though he was mostly in Hawaii getting the CAO up and running, he and his team at Stanford completely sold me on the idea that the future of ecologically relevant remote sensing was multispectral 3D scanning (or better- hyperspectral- but one must start somewhere!). 

I coveted the CAO.   I wanted so much to use it to scan my research sites in China.  Our high-resolution ecological mapping efforts there had been so difficult and the 3D approach seemed to offer the chance to overcome so many of the challenges we faced. 

Yet it still seemed impossible to make it happen- gaining permission to fly a surveillance-grade remote sensing system over China?  It would take years and tremendous logistical and political obstacles to overcome.  So I changed my thinking…

What if we could fly over landscapes with a small hobbyist-grade remote controlled aircraft with a tiny LiDAR and a camera?  Alas, no, - LiDAR systems (high grade GPS + IMU) are way too heavy, and will be for a long time.

Then I saw Photosynth, and I thought- maybe that approach to generating 3D scans from multiple photographs might allow us to scan landscapes on demand without major logistical hassles?  The answer is yes, and the result, translated into reality by Jonathan Dandois, is Ecosynth.

Can Ecosynth achieve capabilities similar to CAO?  Our ultimate goal is to find out.   And make it cheap and accessible to all- as the first “personal” remote sensing system of the Anthropocene.

This camera looks like it might be a great new camera to try in aerial and ground ecosynth work (note: 2 11 MP photos per second):

GoPro launches HD Hero2 helmet cam, announces video streaming Wi-Fi pack for winter

List of HD HERO2 Feature Enhancements:
• Professional 11MP Sensor
• 2x Faster Image Processor
• 2X Sharper Glass Lens
• Professional Low Light Performance
• Full 170º, Medium 127º, Narrow 90º FOV in 1080p and 720p Video
• 120 fps WVGA, 60 fps 720p, 48 fps 960p, 30 fps 1080p Video
• Full 170º and Medium 127º FOV Photos
• 10 11MP Photos Per Second Burst
• 1 11MP Photo Every 0.5 Sec Timelapse Mode
• 3.5mm External Stereo Microphone Input
• Simple Language-based User Interface
• Compatible with Wi-Fi BacPac™ and Wi-Fi Remote™
- Long Range Remote Control of up to 50 GoPro Cameras per Wifi Remote
- Wi-Fi Video/Photo Preview, Playback and Control via GoPro App
- Live Streaming Video and Photos to the Web

WiFi for Ecosynth?  Amped Wireless Unveils Its Professional Series High Power Repeater and Access Point with Wi-Fi Coverage Range of Up to 1.5 Miles

Amped Wireless unveils its professional long-range Wi-Fi Access Point and Repeater that provide Wi-Fi coverage for large homes, offices, building-to-building applications, open outdoor areas, boats, marinas, RVs and parks.

CHINO HILLS, Calif., Oct. 24, 2011 /PRNewswire/ -- Amped Wireless, the leading manufacturer of high-power, long-range wireless products for the home and office, adds to its successful line of Professional Series High Power Wi-Fi Solutions for indoor and outdoor applications. All Amped Wireless Professional Series High Power Wi-Fi Solutions feature a weatherproof enclosure for indoor and outdoor use, an advanced 600mW amplifier for professional Wi-Fi range, and a high-gain directional antenna to achieve strong, wireless connections up to 1.5 miles. Amped Wireless Professional Series

First of all, Jonathan let me know that removing the duplicate x,y,z coordinates from the GPS data set to run in splines.py to generate the constants to be used in the Helmert transformation was ok. The data that was generated when these duplicates were removed worked fine, the 7 paramaters that were generated were very close to that of the test data. I was given another code that would use those generated parameters to actually transform the point cloud points and make them viewable using the software. It looks like the transformations were correctly done by the apply_helmert code when they were compared to the map of campus. Now that I have results from those transformation programs, my next job is to be able to find the orange buckets that were scattered throughout campus when the scan was taken. 

In order to be able to find a specific color in the generated point cloud the current plan is to scan through all of the data looking for a specific RGB range. All of those points would most likely be accumulated into an array. Then it would have to scan through that array of colored points to find sets of points that are close together. Close together would probably be the magnitude of the distance would have to be less than a certain amount. There's supposed to be multiple buckets in the point cloud and we wouldn't want points from one bucket jumping in with the points from another bucket over a 100 feet away. The idea is to have the code scan through this array of points and if the distance between two of those points is small enough, they would go in the same array. If they're too far away from each other, they would go in seperate arrays. Each array would probably have to have more than a certain number of elements in it in order to be counted. This should get rid of extraneous points that don't actually represent a bucket. After all of this is done, for each array, which should have a group of in this case orange points, we would take the average of all of their x,y,z components to find the center. The output would be a list of x,y,z coordinates indicating the center of what is expected to be a bucket. This is basically just the plan right now. I still need to figure out what ranges RGB would be suitable for this and write the code itself.

The Gatewing UAV mapping company was recently acknowledged as the 'Most Disruptive Innovator' at the 2011 Intergeo conference in Germany.  Gatewing sells very high-end solutions for automated RC aircraft mapping with digital cameras, automatically generating orthophotos and DSMs at high resolution.  While the software is proprietary, I have no doubt that it is grounded in both photogrammetry and computer vision.

I have to agree with the award, it is given out "for the innovative application of existing technologies that will cause a significant shift in the market." (Gatewing email, 10/18/2011).  I think that the Ecosynth computer vision approach to ecological remote sensing also represents a disruptive new technology for the research community along those same lines.  Ecosynth is an application of existing technologies that will allow individual ecologists the ability to make very high-quality 3D color scans of landscape vegetation.  The Ecosynth approach means that field researchers can make useful and accurate measurements of study sites using remote sensing, without the need for fixed wing aircraft fly-overs, at relatively low cost, and without being remote sensing experts.  

Now, if only I had realized that the Gatewing office is in Ghent during my family trip in Belgium earlier in the month, 'D'oh!'

Researchers at the Universität Stuttgart, Institute for Geophysics in Stuttgart Germany, have used manually flown Mikrokopters and semi-automated photogrammetric software to generate high resolution photo mosaics and digital terrain models of a landslide area for tracking terrain displacement.  

An article published this spring in the journal Engineering Geology demonstrated the value of using remote controlled aircraft and off-the-shelf digital cameras for high resolution digtial terrain modeling.  The researchers used photogrammetry and computer vision software VMS to make 3D terrain models with aerial images and compared the results to aerial LIDAR and TLS terrain models.  A network of ~200 GPS measured ground control points were used to assist with image registration and model accuracy with good results.

The authors appear to agree with our sentiments that RC based aerial photography and 3D scanning has the benefits of low-cost and repeatability compared to traditional fixed wing or satellite based data collections.

Unlike our research, the authors of this study were interested in only the digital terrain model (DTM) and vegetation was considered noise to be removed for more accurate surface modelling.

Again...just one more reason for me to get cranking on that next paper!

Image source: http://commons.wikimedia.org/wiki/File:Super_sauze_landslide.JPG

I just stumbled on a great looking globe based data visualization tool: WebGL Globe.

The screen cap at right is from a browser based visualization of 1990 global population data.  While not realted to Ecosynth, this is a really cool technology that could be valuable for other research in our lab, like visualizing global data for understanding global relevance of locations and studies, the GLOBE project.

Some of the example global datasets are fun, Google technology user group meetings or blogger mood, but it would be interesting to see some more ecologically relevant data plotted this way.  For example, trends in estimates of forest cover or urban growth.

Can't wait to see this in 3D, or maybe in a holographic projection!

I have been working on getting to know the python spline code Oana and Jonathan wrote last semester and I've been making some progress. I am mostly familiar with the Linux terminal so Jonathan showed me how to use the Windows DOS command line. We got the test data working with the spline code and the results matched the corresponding flight path pretty well. I was given two other data sets to work on and I have only been able to get one working. The second set contained duplicates of two GPS coordinates which seemed to throw off the scipy function splprep(). In order to make sure this was the case, I ran the code with out the duplicates, and the code ran all of the way through as it should. I'm not sure how removing the duplicates would affect any future steps in generating the 3d image, so I need to find another way to get the code to run.