What We Do

Since this is our first general distribution newsletter in a few years (see the introduction), I thought it would be a good idea to review a bit of the history of GeoCue Group Inc. and what we do.

 

We formed the company under the name NIIRS10 (that’s another story!) in July of 2003.  Thus this is our 14th birthday!  Prior to forming this company, I was the CEO of Z/I Imaging, a global photogrammetry hardware and software company.  Z/I was acquired by Intergraph in 2002 (the point at which I left the company).  Intergraph was, of course, subsequently acquired by Hexagon, the new home of Z/I Imaging.  Our core group of developers joined me from Z/I and thus have a long and rich history in developing advanced photogrammetric and LIDAR hardware and tools.

The first product set of GeoCue (at that time, NIIRS10) was GeoCue, a set of enterprise workflow management tools aimed at organizations who need to manage professional LIDAR and photogrammetric workflows.  GeoCue was rapidly adopted by most LIDAR production companies (in those years, primarily airborne laser scanning, ALS) in North America where it remains strong to this day.

Most of our LIDAR workflows encompassed the Terrasolid processing toolset.  This caused us to form a close relationship with Terrasolid OY of Finland.  In 2005, Terrasolid asked us to become their North American distributor, providing software, training and maintenance.  This, too, we maintain to this day.

In 2009, we recognized that LIDAR was being exploited, to a small degree, by end users of LIDAR data such as federal, state and local governments.  We quickly entered this business by acquiring QCoherent Software LLC of Colorado Springs, Colorado.  QCoherent was the company that initially developed LP360, an ESRI extension that allowed advanced exploitation of point cloud data directly in an ArcGIS desktop environment.  We quickly internalized this product and developed a robust standalone version for use outside ArcGIS.  Since that time, LP360 has become the standard for LIDAR exploitation (especially QC) in a great many agencies including the USGS, the USACE, the USDA, Forest Service, many water management districts and so forth.  LP360 is the desktop LIDAR tool against which all others are judged.

In 2012, we entered the small unmanned aerial systems (sUAS) or “drone” business with the aim of bringing tools such as LP360 to bear on the point clouds produced by dense image matching.  After a very careful evaluation (from an accuracy and model conformance point of view) we selected Agisoft PhotoScan and Pix4D Mapper as the tools of choice for point cloud generation.  We developed agreements with these two companies and began sales and support of these products.  We also started down the path of adding powerful sUAS mapping tools to LP360, such as volumetric analysis.

In 2014 we began development of a global navigation satellite system (GNSS) post-processed kinematic (PP)K) direct geopositioning system for our high-end drone, the AV-900.  This resulted in the AirGon Sensor Package (ASP), one of the most accurate positioning systems for rotary wing drones. The ASP is based on a Septentrio GNSS engine specifically designed for sUAS operations.  Since we were effectively entering a new market, we decided to launch this through a wholly owned subsidiary, AirGon LLC (www.airgon.com).  We also developed an Amazon Web Services (AWS) drone products hosting site called AirGon Reckon.  Reckon serves as a data repository and information sharing portal for companies with multiple mapping sites (particularly the aggregate mining industry).

In 2015, AirGon began offering limited drone services for companies who were testing the waters of drone-based mapping but did not want to internalize the operations until the workflows and products were proven.  Our aim is not to compete with our customers, of course, but to evangelize this game changing technology.  We were early to receive an FAA 333 exemption and had certified Part 107 Remote Pilots a few days after that new rule went in to effect.  We have now flown over 600 missions with a wide variety of potential drone users from aggregate mining to paper mills.

Since we have been in business, we have always done some amount of bespoke software development when that development advanced our commercial products.  These bespoke activities ranged from funded additions to our core software all the way to custom workflow solutions.  An example is a very high throughput LandSat change detection system developed for MDA Information Systems.  This system is controlled and managed by a GeoCue Distributed Processing System.

In 2015, we entered into an agreement with Teledyne Technologies (specifically Teledyne Brown Engineering) to develop an Amazon Web Services based system to manage and disseminate data from their Multi-User System for Earth Sensing (MUSES), a multi-sensor host platform mounted on the International Space Station.  This development, the Earth Sensor Portal (ESP), is being offered as a commercial hosting platform for data from satellite imagery to LIDAR data.

We saw a resurgence of interest in mobile laser scanning (MLS) in late 2016/early 2017.  Our MLS software suite had a bit of a hole in that we did not have a package specifically aimed at asset collection from MLS data.  We have been aware of the Orbit offerings from Orbit GT of Belgium for some time.  Earlier this year, we re-evaluated their offering and signed on as their North American distributor.

 

So this brings us to the present!  We now focus on several business areas:

LIDAR – Both Production and exploitation.  Our portfolio includes:

  • The GeoCue product family for workflow management from production to QC.
  • The Terrasolid family of products for industrial strength geometric correction and processing of both ALS and MLS data.
  • LP360 for high performance data editing, QC and specialized functions such as hydro modeling on the product side and a rich exploitation environment (in ESRI and standalone) on the LIDAR consumer side.
  • Orbit GT for feature extraction from MLS data.
  • LIDAR Server for local data management/distribution.
  • Earth Sensor Portal for enterprise data hosting and dissemination.

sUAS (Small Unmanned Aerial Systems) – Data collection, processing and management.  These offerings are via our AirGon subsidiary.  The main thing to remember about AirGon is that we can bootstrap you into the drone mapping business and we can do it in a very cost effective yet completely professional manner.  Our offerings include:

  • Agisoft PhotoScan, Pix4D Mapper – software for generating point clouds and orthomosaics from drone collected imagery.  We offer the software, support and training.
  • LP360 (sUAS licensing level) – The same LP360 as used for LIDAR.  We offer it at a lower price point for small area mapping, such as with drones.  LP360 provides the full workflow from point cloud ingest (point clouds from imagery and LIDAR) to derived product output.  This is really the most powerful tool kit on the market for processing drone data where the desired outputs are high accuracy mapping products.
  • Bring Your Own Drone (BYOD) Mapping Kit – this is a kit of PhotoScan, LP360, a Reckon subscription and comprehensive training that enables data production from any drone.  It has a special emphasis on mapping using low cost DJI drones such as the Inspire 2 or the Phantom 4 Pro.  The BYOD plus a DJI is a great starting point for entering the drone mapping business and we train you in how to be successful.
  • AirGon Reckon – our AWS-hosted data dissemination system for drone data.  This is a great tool for service providers who want to deliver data in a professional, cloud hosted manner to their customers.  Service provider partners can actually use Reckon as a revenue generator for their business.
  • LOKI – This is the most exciting product we have developed in the hardware arena in some time.  It allows you to add PPK direct geopositioning to a Phantom 4 Pro, an Inspire 2 or any drone with a camera equipped with a flash hot shoe.  There is a separate article in this newsletter regarding LOKI.  Add this to a BYOD and you truly will have a professional mapping kit with direct geopositioning using low cost DJI drones!  This is a financial game changer.

ESP (Earth Sensor Portal) – ESP is an Amazon Web Services hosted platform for LIDAR, Imagery and related products.  It is a great dissemination platform for agencies who acquire data such as LIDAR and need it securely backed up and made available to stakeholders (including the public) via a web facing portal.  ESP includes the idea of workflow so agencies can have us integrate in, for example, a QC workflow, allowing their collection contractors to post data directly to their ESP portal.   This is a subscription model that offloads all need for server technology as well as concerns of your own firewall being maliciously penetrated via your data portal.  This is exciting stuff!!

Bespoke Solutions – GeoCue continues to offer custom development when it adds value to our strategic product portfolio.  For example, if you need a niche tool added to LP360, consider discussing a bespoke addition with us. It will show up in the standard code base, maintained as part of the global product.  This prevents you from getting stuck in the situation of having to contract specifically for updates.  On the larger side of the equation, we have developed very large projects for various clients, primarily around LIDAR/imagery data processing, management and dissemination.

 

As you can see, we have a complete product set for several different imagery/LIDAR related production and exploitation scenarios.  We are very happy to entertain your inquiries ranging from simple product questions to those difficult things you encounter in your workflows.  So please keep us in mind when you are thinking of adding a workflow or improving the ones you have.

 

FacebookTwitterGoogle+LinkedInShare

Drone Mapping – Business Models Revisited

I am currently attending the 2017 NSSGA/CONEXPO exposition.  One of the keynotes from the National Stone, Sand and Gravel Association (NSSGA) conference focused on the rate of change of technology in the mining industry and the scope of operations that are covered by these technologies.  Of course, one of the examples was the use of drones.  The gist of the discussion was that some of these technologies are in their formative stages; we do not yet fully appreciate the scope of operational affect they will have but to prosper, knowledge of these systems must be internalized.

One thing is very clear – frequent and repetitive mapping will be required to support the automated machinery that is now appearing on advanced sites.  You cannot program a haul truck for autonomous operations if you do not know the location of the road!  Complicating this issue is the fact that the road location changes nearly daily due to the operation itself.

This future trajectory says that mine site mapping will need to become an internal operation.  It will be impractical from both a logistics and cost perspective to outsource drone mapping services.  A second strong consideration is the rapidity with which drone technology is changing.  I think amortizing the cost of a drone over more than 12 months is just not realistic.

Drones are simply platforms for cameras and other sensors (for example, profilers, laser scanners and so forth).  A drone without a sensor is a fun toy to fly but it is not going to have much use in operations!  I am very excited about new platforms from commercial drone companies (mostly DJI).  These new drones include decent cameras in that they now incorporate larger sensors and hybrid shutters.  You can do a reasonable job of mapping with these yet still use them for inspection videos.

DJI Inspire

So I think what we are seeing is the beginning of the end of the purpose-built drone.  You will be able to purchase drones from DJI (and perhaps others) that are nearly a consumable.  You can use the same drone for inspections as you use for mapping.  This is a very important consideration since this greatly simplifies the training of users.

The bottom line here is this – we are seeing the beginning of drones as an everyday tool for mining, industry and construction.  The proper model is going to be internal control of not only flying the systems but also processing the data.  When you need a quick check of a pulley on a conveyor, you will want an internal staff member to quickly fly the inspection job and post the resultant video.  No need to have a third-party system or contractor involved.  It just complicates the flow and adds expense.  This is really the motivation behind our Bring Your Own Drone (BYOD) Mapping Kit.  It lets you use a low-cost drone such as the DJI Inspire to do serious mapping without a lot of complicated leasing or outsourced data processing arrangements.  It also allows you to use the same platform for inspection that you use for mapping.  Give us a call to see how well this solution will meet your specific needs.

AirGon Partners

We spent a lot of time in November and December of last year (2016) developing a coherent strategy for our AirGon business. As you know from prior newsletters, AirGon LLC is our small Unmanned Aerial Systems (sUAS) subsidiary. We have been developing technology for the past three years aimed at implementing and improving sUAS (or, more commonly, drone) high accuracy mapping. Our focus has been in four major areas:

  • Hardware for RTK/PPK grade geopositioning (the AirGon Sensor Package)
  • Software tools for data processing (Topolyst)
  • Reckon, our Amazon Web Services (AWS)-hosted data management and delivery portal
  • Workflow best practices for project repeatability
  • Production Services for customers who do not want to do their own processing

Addressing the sUAS market is a new challenge for us. There is a surprisingly small overlap between our traditional LIDAR/Photogrammetry marketplace and the new drone business. After a few years in the trenches and hundreds of mapping projects, we are rationalizing our business into three different Partner categories. These are delineated by the type of customer:

Technology Partners – These are customers who purchase technology from us to either use for their own internal operations or to offer services. The technology in our portfolio related to sUAS mapping includes:

  • PhotoScan and Pix4D point cloud generation software
  • Topolyst, our purpose-built point cloud exploitation tool for data from sUAS Laser Scanners (LIDAR) and/or data from dense image matching
  • Bring Your Own Drone (BYOD) Mapping Kit, a collection of software that enables serious mapping with a variety of third party drone hardware from low cost DJI Inspires to professional grade senseFly (eBee) fixed wing drones.
  • Reckon, our Amazon Web Services-hosted site data collaboration and delivery portal. Reckon is a subscription product that allows web-based collaboration between the service provider and end user (who may be one and the same)
  • Various hardware components
  • Consulting services, tailored to needs

Network Partners – The AirGon Network program is an emerging part of our AirGon business. It comprises drone mapping services experts who use our technology for data capture, processing and delivery. Network Partners always interact with their AirGon Network client base via Reckon. We offer regular best practices training, exposure to end-use customers and referrals. We can also provide data processing services to those who wish to focus only on flying. This is a program that requires qualification.

Enterprise Partners – These are end use customers of drone mapping services. An AirGon Enterprise Partner can be as small as a single stockpile yard to as large as a multi-national mining company. Enterprise partners generally engage with us via our CONTINUUM concept, a model that allows a customer to tailor a drone mapping solution that exactly fits their desired business model. For customers who wish to do their own data collection, we offer subscription-based back office processing services. For customers who want to outsource data collection and processing, we link Network partners who are the best match for the desired services and locations.

Please get in touch with use (info@airgon.com) if you are serious about high accuracy drone mapping – we would love to work with you!

What Miners Want

I attended the Commercial UAV Expo in Las Vegas at the end of October.  I gave a talk entitled “Mine Site Mapping – One Year In.”  This talk was on our experiences with performing mine site mapping services with our AirGon Services group.   Our services group is primarily about Research and Development (R&D).  We use our engagements with mining companies to discover the products that they need, accuracy levels and, most of all, how to reliably create these products.  These experiences inform both the development of our technology (the MMK, Topolyst, Reckon, the BYOD Mapping Kit) but also help us develop best practices for both collection and processing.

As I prepared for this presentation, I reviewed the mine site mapping projects we have performed over the past several years to tabulate the products our customers have requested.  These turned out to be, in decreasing order of popularity:

  • Site Volumetrics with a priori base line data
  • Site Volumetrics with no prior data
  • Site contours (“topo”) – 2 foot interval
  • Site Contours – 1 foot interval
  • Time series volumetrics (“borrow pit”)

In every case, the customer desired a site orthophoto.  In fact, they usually want an ortho of the entire site with analytic products of a subsection of the mine site.

I thought in this month’s section, I would review these products from the acquisition and processing point of view.

 Volumetrics with baseline data

I have written a few articles about injecting a priori data into a mapping project.  This is the situation where, at some time in the past, the customer has done a site survey and wants to use these data as the bottom surface of stockpiles.  Their primary desire here is for consistency from inventory to inventory.

An example of this, a large limestone quarry that we fly, is shown in Figure 1.  Here baseline data as well as a reclaim tunnel model have been provided to us as a DWG data set.  The illustration of Figure 1 shows these data being used by Topolyst to create a 3D base surface.

 

Figure 1:  Bottom Data with reclaim tunnel model

Figure 1: Bottom Data with reclaim tunnel model

The primary challenge that we have when receiving a priori data is the accuracy of the data.  We often find that these data were obtained by traditional stereo photogrammetric collection techniques so we do not have a point cloud from which to assess accuracy.  Now, done properly, stereo photogrammetry produces survey grade data.  Unfortunately, much of this a priori data was collected with the surface obstructed by existing stockpiles; in other words, it was not a stockpile free base data mapping.  This means that the stereo compiler had to estimate locations under the existing data.  We find that in most cases, these estimations are simply linear interpolations from one side of the obscured area to the other.  We often find these bottom models extending above the current surface.  It is difficult to tell if the data were incorrectly modeled or if the ground has actually changed from the time the baseline data were collected.

A second big challenge we have with these data are a lack of knowledge by the provider as to the exact datum to which the data are referenced.  We are often concerned with elevation differences of just a few centimeters.  The Geoid model really matters when you are approach survey leveling accuracy goals.  We have found, on more than one occasion, a priori data with an incorrect vertical model.  This usually occurs (at least in the USA) as a result of using the incorrect NAD83 to WGS84 transformation.

Over the past year, we have added a lot of refinements to how Topolyst handles this a priori data.  Those of you who do LIDAR or photogrammetric processing will immediately recognize this as the problem of introducing “breaklines” and “mass points” into a model.  LP360 (Topolyst is just a variant of LP360) has always been a very strong product in terms of breakline modeling.  We have added a few features in this area to improve the modeling as it typically applies in UAS mapping.  We are now at the point where we really do not have any software issues with this sort of modeling but the interpretation problems will always remain.

This type of modeling requires:

  • Direct geopositioning (RTK/PPK) on the drone
  • Multiple surveyed check points on the site for data validation
  • Strong modeling tools such as Topolyst
  • A conference or two with the customer to understand the models
  • A lot of patience when defining stockpiles

Volumes with no a priori data

Here the customer is interested only in the volumes of the piles, without regard to location.  The deliverable is generally a spreadsheet with volume, material type, density and tonnage.  Of course, our customer deliveries are via our cloud data platform, Reckon, so we want the toes to be correctly georeferenced.

If you leave out the correct georeferencing (meaning you compute the volume of the pile but do not necessarily try to align it with an existing map), you have the sort of processing offered by a myriad of web-based solutions such as Kespry.  Under this business model, you typically upload the raw drone images which have been georeferenced by the navigation grade GNSS for x, y and the drone barometric altimeter for elevation.  This typically provides horizontal accuracy on the order of several meters and vertical accuracies at about 5 meters.  So long as the camera is properly calibrated, this methodology leads to volumetric accuracies that are accurate to within about 5%.

We never do these projects without some check points.  These are surveyed image identifiable points that we use to check horizontal and vertical accuracy.

The biggest issues we have encountered with this type of project is the definition of the stockpile toe – it is somewhere between comingled piles, it traces along an embankment such as the pit, the stockpile is in a containment bin and so forth.   There requires a lot of careful toe editing in a three dimensional visualization environment such as Topolyst.

We never have issues with accuracy because we always fly with a direct geopositioning system.  For our MMK, it is a Post-Process Kinematic, PPK, GNSS system.  For the senseFly eBee, it is an onboard RTK system.  We always lay out some checkpoints for project verification.

A very clean mine site with stockpiles sitting on a surface is nearly non-existent (except in our dreams).  While you sometimes encounter sites where you can just manually draw a toe, these sites are nearly always at inventory transfer locations, not working mines.  In fact, of all the mine sites we have surveyed, we have encountered only one “groomed” site (see Figure 2).  Even at this site, the upper left and lower right piles required some disambiguation (wow, that’s a big word!) work to separate the pile edge from encroaching vegetation.

Figure 2: A "groomed" inventory site

Figure 2: A “groomed” inventory site

 Site Contours (“topo”)

A surprising number of customers want contours.  As you know, these are elevation isolines at a particular interval.  Most customer want either 2 foot or 1 foot contour intervals.  These data, in DXF or DWG format, are used as input to mine planning software.  I find this a bit odd since I would think by now that this downstream software would directly ingest a LAS point cloud or at least an elevation model.

Contours are always absolutely referenced to a datum (a “Network”).  This can be a local plant datum or, much more commonly, a mapping horizontal and vertical datum such as a state plane coordinate system for horizontal and NAVD88 with a specific geoid model for vertical (at least in the United States).

You can tie to the datums using either direct geopositioning with onboard RTK/PPK or you can use dense ground control points.  I personally would never collect data that must be tied to a datum without having a few image identifiable checkpoints.  Unfortunately, this means that you will need at least an RTK rover in you equipment kit.

A good rule of thumb for contours is that the accuracy of the elevation data should be at least three times the accuracy of the desired contour interval.  This says if you are going to produce 1 foot (30 cm) contours, you need 4” (10 cm) of vertical accuracy relative to the vertical datum.  When you measure your checkpoints, don’t forget to propagate the error of the base station location (which you might be deriving from an OPUS solution).

Preparing a surface for contour generation is perhaps the most tedious of mine site mapping work.  It is generally the only site mapping you will do that requires full classification of ground points (the source for the contour construction).  An example of 2 foot contours within a mine site is shown in Figure 3.

Figure 3:  An example of 2' contours

Figure 3: An example of 2′ contours

Sites with a high degree of vegetation in areas where the customer wants contour lines will have to be collected with either manual RTK profiling (very tedious!) or with a LIDAR system.  You simply cannot get ground points with image-based Structure from Motion (SfM).  No surprise here – this is why LIDAR was adopted for mapping!

If the customer does not want to pay for LIDAR or manual RTK collection, the vegetated areas should be circumscribed with “low confidence” polygons.  You can either exclude the contouring completely from these areas or classify the interior to vegetation and let the exterior contours just pass though the region.  In any event, the customer must be aware that the data are quite inaccurate in these regions.

The SfM algorithm gets quite “confused” in areas with overhead “noise” such as conveyors and vegetation.  This confusion (actually correlation errors) typically manifests as very low points.  You will need to find and clean these points prior to contour generation.

Conclusions

Product generation for UAS mapping requires a lot of front-end planning.  This planning needs to be product-driven.   If you customer (you, yourself, perhaps) needs only volumes with no tie of the toes to a datum, you can get away with no control so long as some other information such as camera calibration and flying height are correct.  By the way, we recommend never collecting this way since you are precluded from doing any meaningful time series analysis.

On the other hand, most meaningful data (that is, you can quantify the accuracy relative to a datum) will require a very careful control strategy as well as a rigorous processing workflow with the right tools (meaning Topolyst, of course!).  No matter what geopositioning strategy you employ, you should always have some independent methods for verifying accuracy.

If all of this seems a bit daunting, you can get assistance from us.  Remember, our services group is really our R&D lab.  Our real goal is to sell technology to owner/operators and production companies.  No matter what drone you are using, you can always avail of our consulting services.  We have gained a lot of experience over the past few years, mostly by first doing the wrong thing!  Save yourself this time and money by engaging with us!

 

 

 

AirGon BYOD Mapping Kit

I am excited to give you a preview of the AirGon Bring Your Own Drone (BYOD) Mapping Kit.  What better way to introduce a small, low cost approach to mapping than with our BYOD Marketing Rep, Molly.  OK, OK a bit of nepotism – she is my granddaughter.

airgon-byod_marketingrep

The BYOD Mapping Kit is a collection of software and training that allows you to do mapping with a low cost DJI drone.  Currently the Phantom and Inspire platforms are supported with the new DJI Mavic soon to be added.

The BYOD Mapping Kit includes:

  • Map Pilot for DJI – Autopilot software for your DJI drone (iOS device required)
  • Agisoft PhotoScan for creating ortho mosaics and 3D point clouds
  • AirGon Topolyst for checking accuracy, adjusting/cleaning data and generating analytic products such as hill shades, volumetrics, digitized mapping features, profiles, topographic contours and similar products
  • A three month Level 1 Subscription to Reckon, our Amazon Web Services hosted analytic data management and delivery portal
  • Web Training
  • Monthly training webinars restricted to AirGon mapping customers

The kit is priced at US $7,990.  Just add your own low cost DJI platform and you are in the mapping business!  This is a great way to get your feet wet with drone mapping.  While this kit is suitable for service providers who want to start out with a conservative approach to drone mapping, it is also a great way for owner/operators to experiment with the viability of this approach to data analysis.  For example, we have been working extensively with a paper mill who uses the BYOD with an Inspire to produce volumetrics for wood chip and log piles.   Another example is an asphalt shingle company who is measuring volumes of raw and processed shingles with a Phantom.   Now granted, you are not going to collect accurate 1 foot contours with a Phantom but you can do some serious analytics that are good enough for many estimation purposes.  This initial BYOD Mapping Kit requires an Apple iOS device (iPhone or iPad) for the autopilot.  We will be adding an Android option by Q1 of 2017.

Of course, you could assemble this yourself by individually acquiring the components.  However, the most important aspect of the BYOD Mapping Kit is the on-going training you will receive as a member of the program.  We have performed over 250 site mapping projects in the past year and have learned what does not work (almost everything!) and what does work.  For example, you will not get anything close to correct without a process for focal length determination (no, it is not what is written on the lens!), elevation bias removal and a number of other tricks.  Our paper mill customer was experiencing volumetric errors of around 25% using a mainstream point cloud/basic volumetric tool prior to engaging with us.  We did some diagnostics on the process and improved their accuracy to within 5% of reference (reference was a very high accuracy survey conducted by AirGon using our AV-900 helicopter with PPK and survey ground control).  In fact, one of the errors is a transformation problem within the DJI recording software itself.  I can assure you that the BYOD Mapping Program will provide a very rapid Return on Investment via the training alone.  If you decide to move up to a survey grade drone, the PhotoScan and Topolyst software remain the best possible solution.  Thus your total investment is preserved as you migrate to more capable systems.

If you just want to collect data but do not want to do routine data processing, no problem.  We can do direct data processing for you via our AirGon Services Group or direct you to one of our AirGon Partner Program members.

If you are interested in becoming an AirGon BYOD Mapper, contact Ashlee Hornbuckle at ahornbuckle@airgon.com.  She will be happy to share detailed information on this program with you.

AirGon Happenings

This has been a very busy time for our AirGon subsidiary.  While our primary focus is delivering hardware and software tools for high accuracy drone mapping, we also provide a limited amount of services.  These services have been extremely helpful in providing a test bed for our positioning and processing tools.

We continue to test and provide feedback to our LP360/Topolyst software development group regarding tools for improving the overall workflow experience.  We run in to all sorts of complex modeling situations and we try to assess each in terms of tools that would ease and/or improve the workflows.  For example, you will see a new tool to extract 3D vertices from line work in the latest EXP release of Topolyst/LP360 (standalone).  This tool has been added to assist with modeling Low Confidence Areas (LCA) common to point clouds derived from Structure from Motion (SfM) algorithms.

Another recent edition to LP360/Topolyst (all versions) is a new contour smoothing algorithm.  This algorithm is designed to address the problems of meandering contours in areas of small vertical change (the meanderings are caused by either surface or algorithm noise).  You will find that this new tool greatly enhances the appearance of contours in these problem areas.  A typical meandering contour is depicted in Figure 1.

A typical meandering contour

Fig 1: A typical meandering contour

This same map area after processing through the new smoothing algorithm is shown in Figure 2 – a dramatic improvement!  Our algorithm works in model space (the model on which the contours are based) and hence is guaranteed not to introduce topology errors such as contour crossings.

Figure 2:  Contour processed through LP360/Topolyst smoothing algorithm

Figure 2: Contour processed through LP360/Topolyst smoothing algorithm

We have been doing a lot of experiments lately with very low cost drones and cameras (for example, the Inspire Pro) as to their suitability for volumetric mapping.  The results so far are mixed.  We have discovered that, when using no control (an approach often used by folks not well versed in survey grade mapping) that an error in the a priori heights fed into SfM software will result in significant scale errors.  These scale errors are not immediately evident since all of the data look terrific!  I hope to be publishing a report on this within the next 60 days.

We did our first flights under the new Part 107 rules.  We were collecting data near an airport in Class G airspace (something we could not do under the old Section 333 waiver without a special COA).  We always carefully monitor air traffic via a VHF radio.  At one point we heard a pilot declare “I see a drone down there over the mine site!”  This is perfectly OK under Part 107 but takes a bit to get used to!

We are concluding that if you need point clouds from imagery (dense image matching, DIM or Structure from Motion, SfM) to meet the network accuracy requirements for high grade topographic mapping (such as 1 foot contours) you are going to have to use either RTK or PPK on your flight platform.  Even with fairly dense ground control, we are not seeing the accuracy levels we need without RTK/PPK (and we have tried this with different systems and cameras).

We are considering a special training session later this year on drone data workflow processing using PhotoScan/Pix4D and Topolyst.  We also may work with our local flight center to combine this with a Remote Pilot certification training/testing session.  Drop us a line if you are interested in this.

 

 

Creating Stockpile Footprints in Topolyst

Several months ago, I introduced Topolyst, our small Unmanned Aerial Systems (sUAS) processing software.  One of the great features in Topolyst are tools to automatically create the footprint (“toe”) of a stockpile and to optionally classify overhead points so that they are excluded from subsequent processing (such as cross sections or volumetric computations).  An example of a stockpile with an overhead conveyor, prior to toe finding and classification, is shown in Figure 1.  As seen in the 3D view in the upper right, the conveyor simply blends in with the stockpile, giving a grossly inaccurate volume for this pile.

A typical stockpile with overhead conveyor

Fig 1: A typical stockpile with overhead conveyor

The data following Topolyst’s automatic stockpile extraction are shown in Figure 2.  Note the toe in the Map and 3D views as well as the automatic classification of the portion of the conveyor within the toe.  This is an extremely powerful tool available in Topolyst[1] (or in LP360 Advanced) that reduces the work of collecting stockpile volumes significantly.  Our initial release of Topolyst also includes a very powerful collection of 3D feature editing tools that make quick work of manually digitizing toes or cleaning up toes in difficult locations (for example, along pit walls) following automatic extraction.

Figure 2:  Automatically extracted stockpile with overhead classification

Figure 2: Automatically extracted stockpile with overhead classification

We have found, from completing many stockpile surveys, that correctly defining the toe is just the beginning!  Mine site operators are keenly interested in consistency.  For example, suppose a stockpile is measured on 5 January to have a volume of 1,000 yards3.  The plant manager sells 500 yards3 from this pile during the period up to the next survey.  She also estimates that 1,000 yards3 were added to the pile.  The next survey should indicate a volume close to 1,500 yards3.  If it does not, the person measuring the volume is the first suspect!

What are the causes of these discrepancies?  The first is, of course, poor estimation.  It is much more difficult to accurately estimate the volume of a pile by “eyeball” than one might guess.  However, we have found the primary culprit to be the definition of the base of the stockpile.

Many mine sites keep a priori survey data that represent the terrain prior to placing any stockpiles (“baseline data” or simply baselines).  Nearly all of the baseline data provided to us has been stereographically collected from a manned aerial survey.  An example is shown in Figure 3.  The magenta points are 3D “mass points” that were derived from a conventional photogrammetric stereo model.

Figure 3:  Baseline data (magenta points) superimposed on a shaded relief of the site

Figure 3: Baseline data (magenta points) superimposed on a shaded relief of the site

The question arises as to how to consistently employ these baselines?  There several approaches that one can take:

  • Get the mine site owner to agree to use the true surface at the time of data collection and abandon the use of “baseline” data. There is a lot of argument for this since it is seldom that the subsurface material will be used.  However, a big one time inventory adjustment may have to be made.
  • Use the 3D toes to define the vertical edge of a stockpile but pull down the base geometry using the baseline data
  • Generate a surface model from the baseline data and then use the toes to only define the planimetric placement of the stockpile.

The third method probably gives the most consistent change of volume record from survey to survey but is it the most technically correct?  This method assumes that all of the material from the toe to the baseline (recall that the baseline is actually under the surface on which the toe lies) could be extracted and used/sold.  This is usually not the case.

As mappers of data, it is important that we advise mine site operators of the advantages and disadvantages of the various methods but, at the end of the day, produce the data according to the customer’s instructions.

Topolyst supports all of the aforementioned techniques for computing volumes (as well as a few others).  For example, the hillshade of Figure 4 is a surface model constructed solely from photogrammetric mass points.  Topolyst has the ability to dynamically use these data as the base where computing volumetrics.  Topolyst also has the ability to generate a LAS file from point, polyline and polygon feature data.  This is extremely useful since this “baseline” LAS can be used in a wide variety of analysis scenarios.

Figure 4:  A surface model constructed from photogrammetric mass points

Figure 4: A surface model constructed from photogrammetric mass points

The features we are adding to Topolyst are being driven by our customer needs, our own needs within our analytic services group and by our research and development efforts aimed at process improvement.  I very definitely welcome your feedback on current and needed features in this great product.


[1] LP360 Advanced (standalone) is feature equivalent to Topolyst

AirGon Happenings

I am pleased to announce that AirGon’s request for amendment to its Section 333 waiver for flying commercial small Unmanned Aerial Systems (sUAS) was approved in April.  Our amendment adds all current and future 333 approved aircraft to our 333.  AirGon can now fly any sUAS that has ever been approved by the FAA as well as all future approved systems.  This list currently contains 1,150 different sUAS (AirGon’s own AV-900 is number 207 on the list).  This provides us a lot of flexibility in working with clients; for example, in situations where a glider sUAS is more efficient than a rotor craft.

The FAA has also recently streamlined the process of obtaining an N number for a sUAS.  Prior to the change, a paper process that required several months was the only option.  Now an online system is available, greatly simplifying this procedure.  Note that this is not the new online registration system for hobby drones but rather the system used for obtaining an N number for a manned aircraft (if you are confused, join the club!).  Combined with our new 333 amendment, we can now get a new aircraft legally operating within days.

We continue to do a lot of work to optimize the accuracy of point clouds derived from dense image matching (DIM).  DIM are the data of choice for sUAS mapping since they can be generated from low cost prosumer cameras using standard application software such as Pix4D Mapper or PhotoScan.  The question always remains as to how good these data really are.

It has taken us a lot of experimentation and analysis but we think we have fleshed out a procedure for assuring good absolute vertical accuracy.  It involves the use of Real Time Kinematic (RTK) Global Navigation Satellite System (GNSS) positioning on the sUAS, a local base station that we tie into the national Continuously Operating Reference Station (CORS) network and the National Geodetic Survey’s Online Positioning User Service (OPUS) to “anchor” the project to the network.  We have also discovered that high vertical accuracy cannot be obtained without camera calibration.  We typically use an in situ process for calibration.  We have flown many dozens of sites (primarily mining), giving us a rich set of test data.

I cannot over emphasize how critical network vertical accuracy is.  Most customers want elevation maps of their sites.  These are usually delivered as contour vector files.  As we all know, a 1 foot contour requires vertical accuracy of 1/3 of a foot.  This is a very tight requirement!  A three inch vertical bias error over an acre is an error of about 400 cubic yards – this is significant.

We see a lot of drone companies processing site data with no control and no RTK/PPK.  While, with the introduction of scale into the model (many companies do not even do this), one might obtain reasonable difference computations (such as volumes), the network accuracy is very poor (obtained from the airborne navigation grade GNSS only) and hence the data are of limited use.  We have discovered that these techniques (where no control and/or RTK/PPK is used) can result in the vertical scale being incorrectly computed.  This means that even differential measurements are not accurate.  Why spend all of the money to collect these data if they are of unknown accuracy?

A more difficult area that we have studied over the past several years is what I refer to as “conformance.”  That is, how well does the DIM actually fit the object being imaged?  DIM processing software (again, such as Pix4D and PhotoScan) do a miraculous job correlating a 3D surface model from highly redundant imagery using the general class of algorithm called Structure from Motion (SfM).  In addition to the obvious areas where SfM fails (deep shadow, thin linear objects such as poles and wires), a lot of subtle errors occur due to the filtering that is performed by the SfM post-extraction algorithms.  These filtering algorithms are designed to remove noise from the surface model.  Unfortunately, any filtering will also remove true signal, distorting the surface model.

We are working with several of our mining customers to quantify these errors and, once these errors are characterized, to develop best practices to minimize or at least recognize when and where they occur.  An example of an analysis is shown in Figure 1.  Here we are analyzing a small pile (roughly outlined in orange) of very coarse aggregates with a volume of about 340 cubic yards.  This site was flown with a very high end manned aircraft LIDAR system and with AirGon’s AV-900 equipped with our RTK system.  The DIM was created using Agisoft PhotoScan.  We obtained excellent accuracy as determined by a number of signalized (meaning ground targets visible in the imagery) control and supplemental topo only shots.  We used in situ calibration to calibrate the camera (a Sony NEX-5 with a 16 mm pancake lens).

As can be seen in Figure 1, we created a series of cross sections over the test pile.  These cross sections were generated using the Cross Section Point Cloud Task (PCT) in LP360/Topolyst.  This tool drapes cross sections at a user specified interval, conflating the elevation value from the user specified point cloud.  We ran the task twice, conflating Z first from the LIDAR point cloud and then from the DIM.   In Figure 1 we have drawn a profile over one of the cross sections with the result visible in the profile view.  The red cross section is derived from the LIDAR and the green from the DIM.

Comparing LIDAR (red) to DIM (green)

Comparing LIDAR (red) to DIM (green)

Note that the DIM cross section (green) is considerably smoother than the LIDAR cross section (red).  This is caused by several factors:

  • The aggregate of this particular pile is very coarse with some rocks over 2 feet in diameter. This leaves a very undulating surface.  The LIDAR is fairly faithfully following this surface whereas the DIM is averaging over the surface.
  • The AV-900 flight was rather high and the data was collected with a 16 mm lens. This gave a ground sample distance (GSD) a little higher than is typical for this type project.
  • Due to the coarseness of the aggregate, significant pits appear between the rocks, creating deep shadows. SfM algorithms tend to blur in these regions, rendering the elevation less accurate than in areas of low shadow and good texture.

The impact of lower conformance is a function of both the material and the size of the stockpile (if stockpiles are what you are measuring).  For small piles with very coarse material (as is the case in this example) a volumetric difference between LIDAR and SfM can be as great as 20%.  On larger piles with finer aggregates, the conformance is significantly better.   For example, in this same test project, we observed less than 0.25% difference between LIDAR and the DIM on a pile of #5 gravel containing about 30,000 cubic yards.

There still remains the question of which is more accurate – the volume as computed from the LIDAR or the volume as computed from the DIM?  I think that if the LIDAR are collected with a post spacing ½ the diameter of the average rock, the LIDAR will be the most accurate (assuming that it is well calibrated and flown at very low altitude).   However, the DIM is certainly sufficiently accurate for the vast majority of aggregate volumetric work, so long as a very strict adherence to collection and processing best practices is followed.  For most high accuracy volumetric projects, manned LIDAR flights are prohibitively expensive.

We continue to do many experiments with local and network accuracy as well as methods to improve and quantify conformance.  I’ll report our results here and in other articles as we continue to build our knowledge base.

Your Business Model, not Ours!!

We have invested a tremendous amount of resources (monetary, development, knowledge) into developing technology and services for mapping sites using dense image matching collected with small Unnamed Aerial Systems (sUAS). Our focus is applications suitable for an sUAS (non-populated areas, smaller sites) that require near survey grade accuracy. The most common example is small open pit mine sites such as quarries. We have not considered agricultural applications since these tend to be very large areas where radiometric analysis is the focus rather than geometric correctness.

Like most other companies involved in this emerging market, we are trying to predict the most palatable business model. However, I would say that unlike many other technology providers, we are seeking the business model that makes the most sense for the customers, not for us.

AirGon LLC has a very big advantage over companies funded by venture capitalists. We are funded both by GeoCue and by investments from our small group of inside shareholders. This allows us to focus on a long-term vision of the market. We plan to become the “go to” company for sUAS mapping, much as GeoCue has become the “go to” company for airborne and mobile laser scanning.

One of the big questions that Venture Capitalists have in funding a startup is that of scale. If the venture will not scale up to a sufficient size to provide a comfortable multiple on the initial investment, the venture is not considered financially viable. In the sUAS business, it is hard to devise a model that will scale that does not require significant involvement on the part of the customer. The most popular model is a leased plan where the customer flies the drone and uploads the image data to a cloud-hosted system provided by the vendor. In some of these models, the customer may even do the data extraction such as defining the base of a volumetric stockpile.

These “self-service” business models proliferate in the rollout of new technologies that are generally called “Web 2.0” (or are we Web 3.0?). You now see it with everything from reservation booking systems to the Uber taxi concept (in the Uber case, the job of “dispatch” has been handed over to the customer). Even grocery stores have gotten on this bandwagon with self-service checkout kiosks.

We certainly believe that self-service will play a major role in the emerging sUAS mapping business. However, at the current time one size does not fit all. This is particularly true in light of the draconian FAA regulations that currently exist for commercial sUAS operations. A mine site leasing a “fly-it-yourself” drone would require an FAA 333 exemption as well as an FAA licensed pilot. This is a fairly significant barrier to adoption of the technology. In addition to the legal hurdles, many customers want to nibble into this new approach to mapping rather than wolf it down in one gulp.

We launched our CONTINUUM program as a way to address these customer needs. CONTINUUM allows a customer to pick from a menu of hardware, software and services that best suit her needs. A few customers want to buy a mapping kit and do it all themselves. For this customer we offer the AV-900 Metric Mapping Kit (in both base and RTK versions). Other customers want to fly their own equipment but have the data processed as a service. Still others want to have us provide full services where our Field Service Analyst shows up at their site and performs the complete job. Under CONTINUUM, we can provide what the customer wants, not what we think might be the best business model for us.

One of the real values behind CONTINUUM (and the reason for the name) is that most customers do not know what they will want to do as a final business model. They would like to be in an environment where they can experiment a bit. This is exactly what we provide through the CONTINUUM program. A customer can modify the business model from one of AirGon doing everything to they, themselves,  internalizing the entire process or any mix in-between, all without the need to change vendors.

I am not sure what will be a profitable business model for AirGon. We are still very heavily in the Research and Development mode. However, one thing I do know for sure – the successful business model will be the one that is deemed successful by the end-use customer. We intend to be the provider of that ultimate solution!

 

 

Top Ten Considerations for Selecting a Drone Mapping Services Vendor

You realize that significant benefits would be realized by transitioning mine site mapping/volumetrics to drones (more properly, small Unmanned Aerial Systems, sUAS). You have decided, at least for the immediate future, to use an outside service provider rather than internalize the process.

Since you have, at least for the present, decided to outsource drone-collected mapping and volumetrics, the task now is to select a qualified company to perform these services. A checklist for evaluating a potential service provide should include these questions:

  • Is the vendor authorized to fly by the appropriate regulatory body (e.g. in the USA, the Federal Aviation Administration requires a Section 333 exemption permitting commercial drone flights)?Drone Picture
  • Does the vendor have sUAS aircraft liability insurance?
  • Are the rights to the collected data clearly spelled out?
  • Do you feel confident that the vendor’s methodology for rigorous network/local accuracy (surveying accuracy) will meet your requirements? For example, a 4 inch vertical error in a borrow pit computation amounts to about 538 cubic yards of volumetric error per acre!
  • For projects that require Network Accuracy (anytime you intend to extract information such as elevation models, contours or are performing time series analysis, you will need Network Accuracy), can your service provider tie their results to a reference network that can be independently verified?
  • Does the vendor have a plan for incorporating surveyed quality assurance check points that will be captured in the aerial flight?
  • Does the vendor understand how to incorporate design information such as “bottom” lines, reclaim tunnel models, complex a priori stockpile toes and so forth into the modeling process?
  • Does the vendor have a reasonable approach to allowing you to collaborate on resolving project boundaries, stockpile identification, stockpile toe definitions, occluded areas and so forth?
  • Have proposed ground personnel worked on mine sites and have safety awareness? For example, for USA mine site operations, do they have basic MSHA Part 46 training?
  • Can the vendor provide references?

You should engage in a pilot project with your candidate vendor. This will limit your initial investment and give you an opportunity to fully vet the proposed provider before committing to a long term relationship. You will want to have independent test data to validate the vendor’s solution.

An immediate red flag is a potential vendor who will not explain their methods in detail, hiding behind a veil of “well, that is our proprietary method that sets us apart from our competitors.” The plain English translation of this is “I have no clue!”