Internalizing LIDAR Data Processing

At GeoCue Group we are involved with customers across the mapping industry, from hardware designers through data collators and data analysts to end users, so we often get asked the question, “how much data processing should I do myself?”  It is a great question.  How much of any given business process you decide to internalize must be a key part of your overall growth strategy.  Unfortunately, we often see companies making one of two classic mistakes when approaching this question about bringing LIDAR data processing in-house.   

The first mistake is to decide to do something just because you can, and being smart engineers and scientists, we all believe we can do LIDAR data processing!  In fact, from a practical point-of-view, this is probably very true.  Most engineering, survey and mapping firms have the technical capability and skills already on staff, or can acquire them by hiring experienced people, to take on LIDAR data processing.  LIDAR data is no more complex than many of the other geospatial data types companies routinely process in-house.  It has some unique aspects to it, but the workflows, tools and techniques are very teachable and can be learned, although there is no substitute for experience.  But, just because you can do a thing does not mean you should do that thing.  For LIDAR data processing, a compelling business case must exist to justify internalizing the process.

Let’s consider the case of a company that is currently subcontracting out all their LIDAR data production.  Typically, they will be receiving geometrically correct, fully classified point clouds as a deliverable.  There are usually two questions such companies ask when looking at what, if any, of that work would be better done internally. First, do we want to and can we afford to get into the data collection business by buying hardware? Second, if we don’t buy a sensor and continue to pay somebody else to collect, how much of the data processing should we do ourselves?  The hardware question is usually driven by larger business considerations than we are discussing here, given the level of capital investment required.  There is also a clear difference between taking on work that involves field data collection and all the logistics that go along with those activities and taking on what is essentially another back-office data processing workflow.  We usually recommend that if you aren’t already doing field work, don’t decide to get into it by starting with LIDAR data collects.  But what to do about the back-office data processing is always an interesting question for any company.  The advantages of bringing LIDAR data processing in-house are often characterized in terms of cost-savings – our subs are charging us way too much! – and schedule control – our subs are always late!

The cost-saving argument can be a strong one, but it requires careful analysis.  When we discuss standing-up a LIDAR data production team of three to five staff, we recommend companies allocate an estimated $65,000 to $95,000 for software licenses, classroom training and updating their IT infrastructure.  The minimum investment, for the smallest operations (single technician, existing IT hardware, limited training) still is going to be in the $20,000 – $25,000 range.  The annual lifecycle cost to maintain this capacity likely will run around 20% per year covering software maintenance, support, and annual training.  So, the five-year capital investment for our 5-person team is going to be around $175,000 or approximately $35,000 per year.  The labor costs are going to be the big variable cost; if you have enough work to keep your new production team busy full-time doing LIDAR data processing, the salary and overhead for a five-person team for the year likely will be significantly larger than your actual capital investment in the software tools.

Unfortunately, it is here that many companies get side-tracked.  They see the large up-front capital investment required for the software and training and struggle to get over that hurdle – because usually someone must be convinced to sign an actual purchase order for this amount! – even though in the long run it is likely the labor costs that will determine the profitability of the venture, not the initial set-up costs.  We often hear from companies that want very detailed breakdowns on pricing and technical capabilities of the software to support their business case but can’t tell us exactly how many people they plan to have working on the data processing or what the annualized labor burden will be.  They focus too much on the software price and not enough on putting the software investment in the context of an overall business case.  Ultimately the actual financial determination in this case is straightforward; if the company is paying more than $35,000 + X per year (where X is the organization’s labor burden based on their projected workload) for LIDAR data processing, they can save money by bringing that data processing in-house.

Control of the data processing, especially schedule control, is the other common justification for internalizing LIDAR data processing.  However, our experience has shown this is often a red-herring.  Poor performance on past projects is more likely to indicate a problem with the choice of subcontractor rather than a process issue.  There is nothing that internalizing LIDAR data processing will do to improve upon best practices. If you do decide to internalize, getting trained on best practices is critical!  We work with the best LIDAR data producers in the world and by applying best practices, being rigorous about workflow management and applying constant quality improvement, they all produce great products on time and on budget.  We firmly believe any company that is willing to invest in the proper software tools and well-trained people can achieve the same results by internalizing the process.  Controlling the data processing does offer the potential to build efficiency improvements into your processes over time that can help reduce delivery schedules, but any credible subcontractor will be doing the same and passing those savings on to their customers anyway.

The second common mistake we see companies make in building their business case for internalizing LIDAR data processing is to delay full implementation or adopt a slow rollout strategy.  LIDAR data processing is one of those activities that benefits greatly from economies of scale and “doing the work.”  Achieving a critical mass of expertise on staff and having a constant workload is very important to a successful internalization program.  Having a plan where staff will work on LIDAR part-time or only at certain times of the year or only on a certain customer’s projects is usually a very high-risk choice.  Even if financially the business case appears strong, we often caution customers that if they aren’t going to truly prioritize LIDAR data processing as a core competency and build a sustainable pipeline of work from Day 1, they may be better off staying with a subcontractor.  Often rather than slowly ramping-up to a successful deployment, they end-up slow-walking down a dead-end path that leaves them with only a bare minimum of internal capability, though having invested heavily in the software tools and training.  In the worst-case scenario, these are the companies that we see exit the LIDAR data processing business after 18-36 months with little to show for their investment.  The best way to mitigate the risk of a stalled or under-utilized deployment is to avoid a piece-meal deployment plan; if the financial business case for internalizing LIDAR data processing is there, then be aggressive!

Get the PDF – Internalizing LIDAR Data Processing

FacebookTwitterGoogle+LinkedInShare

GeoCue Launches GetLidar.com to Support Hurricane Recovery

When natural disasters occur, one of the more pressing needs of disaster recovery teams is access to trust-worthy, pre-event data. Typical needs are for recent aerial images and elevation data, preferably orthoimages and LIDAR data. It can be difficult to find sources for this data, sources that can be easily accessed from any location, trust worthy with respect to data integrity and accuracy, and which provide a simple, straightforward interface to extract and deliver data to local computers for processing. GetLidar.com provides such an access point for data relevant to the areas in Florida, Texas and Puerto Rico heavily damaged by hurricanes Harvey, Irma and Maria.

GetLidar.com provides free and direct access to pre-event imagery and Lidar data including:

  • LIDAR data in both American Society for Photogrammetry and Remote Sensing (ASPRS) LAS format as well as compressed LAZ format.
  • 50 cm orthophotography in US Geologic Survey (USGS) quarter quad format for the Harris county area, provided by the Texas Natural Resources Information System (TNRIS). Collected on behalf of the Houston-Galveston Area Council (H-GAC)
  • 2004 USACE LIDAR and 2015 National Oceanic and Atmospheric Administration (NOAA) NGS Topobathy LIDAR data in LAS or LAZ format from NOAA for Puerto Rico.
  • US Department of Agriculture National Aerial Imagery Program (NAIP) for all areas
  • Landsat 8 data for all areas

Read Complete Article: GeoCue Launches GetLidar.com to Support Hurricane Recovery

The Secret is Out

By Ashlee Hornbuckle

I have been tasked with keeping a secret over the past few months—which is sometimes difficult for me, as I have trouble remembering what has been told in confidence.  However, Lewis told me a secret so big, I almost blabbed for the mere fact of how utterly exciting it is.  Are you ready for it?

AirGon has developed a direct geopositioning Post-Process Kinematic (PPK) system for the DJI series Inspire 2 and Phantom 4 Pro drones!

Dubbed LOKI, this PPK system is a third generation AirGon design that uses the latest Septentrio GNSS engine, the AsteRx-m2.   The m2 is a triple band GNSS engine, supporting NAVSTAR GPS L1/L2/L5 and GLONASS L1/L2/L3 and sporting 448 hardware channels.  The GeoCue engineers tell me this is the most advanced UAS class receiver on the market today.

LOKI is self-contained and uses an internal battery (charged via a USB port).  It has been designed to survive most crashes and easily can be moved to a new, replacement drone.  LOKI interfaces to the DJI series drones by simply plugging a personality cable into the DJI drone SD card slot, making it a user installable “plug and play” system.  We use a patent-pending set of hardware and firmware algorithms to figure out when the camera is triggered.  Should users elect to use a higher end drone with a DSLR camera, the LOKI system can be moved by simply using a DLSR personality cable.

LOKI provides a huge advantage over using drones without RTK/PPK.  Without direct geopositioning, dense ground control is required to achieve the accuracy necessary to calculate differential earth works volumes and to create 2’ (60 cm) or closer contours. This can be extremely time consuming and sometimes a safety issue. In fact, on some sites ground control cannot be placed due to restrictions to site access. However, joined with the BYOD Mapping Kit and a base station, users can expect around ⅛ foot horizontal and ¼ foot vertical accuracies with no ground control placement.

The system is scheduled for release in late July/early August. Please contact us at info@airgon.com for more information on LOKI and the BYOD Mapping Kit.

Terrasolid: The Workhorse is Still a Valuable Tool in LIDAR Production Shops

As the North American reseller for Terrasolid’s software suite, we get to work with the majority of the LIDAR production shops in the US and Canada.  The Terrasolid suite – TerraScan, TerraModeler, TerraMatch and TerraPhoto – continues to be common-place on the production floor regardless of the type: airborne, mobile or terrestrial.  And increasingly we see UAV operators deploying Terrasolid to assist with their own point cloud workflows, whether LIDAR or imagery based.  The focus of the industry is often on the what is new and different and exciting, on the “latest and greatest” so this week we thought we’d step back from the hype and hoopla and check-in with a long-time user of Terrasolid to see how this old workhorse of the LIDAR production shop is doing these days.

We spoke with Amar Nayegandhi, Vice President of Geospatial Technology Services at Dewberry.  Dewberry has been using LIDAR commercially since 1998 – yes, 1998; Dewberry received the first LIDAR task order from the USGS under the Cartographic Services Contract (CSC) – and is well-known and well-respected in the industry.  GeoCue Group sold our first seat of GeoCue and Terrasolid software to Dewberry more than 10 years ago back in 2007.  Dewberry is also a major user of our LP360 software along with many other commercial software tools that are available on the market; basically, they know their stuff when it comes to LIDAR software.

What is the biggest benefit you get from using Terrasolid in your business?

One of the biggest benefits of Terrasolid software is we can integrate the entire LIDAR workflow into our MicroStation CAD environment.  Our geospatial and engineering professionals have a very good understanding of the CAD environment, which enables us to perform point cloud processing (TerraScan), surface modeling (TerraModeler), and sensor calibration (TerraMatch) directly in the CAD environment.

Of the four modules, TerraScan is the primary point cloud analysis tool; where do you see it helping you the most?

When we first started working with LIDAR data, just being able to load millions of points into our CAD software was a challenge that TerraScan solved for us.  Now data sets are in the billions of points and expectations of basic point cloud functionality has evolved with the times.  Still, the core functions we use TerraScan for haven’t changed much over the years – our biggest benefit is the automatic bare earth filtering using our proprietary macros developed through years of experience in processing LIDAR data in various environments. Some of the newer tools in TerraScan like Groups for spatial object classification or newer surface classifications for pulling ground from noisy UAV data are really helping as well.  Project and data management tools are also big time-savers we often take for granted.

After TerraScan, what module do you find the most critical for your production?

Probably TerraMatch.  Sensor manufacturers have come a long way in having calibration and geometric correction built right into their pre-processing software, but TerraMatch gives us the ability to independently verify and correct the fit of the data.  We often use TerraMatch to calibrate data in a project area that include multiple “lifts” because sensor-manufacturer software does not always produce the best fit over lifts that have variable GPS/IMU trajectory solutions. It is also vital for working with older data sets or subcontractor-provided data where we may have no visibility into the calibration processor – TerraMatch gives us an independent verification of goodness of fit.  For mobile LIDAR data, with the GPS outage concerns and other aspects particular to driving around in a car as opposed to flying over in an aircraft, having a set of tools like TerraMatch for calibrating the laser scanners and the cameras is absolutely mandatory.

Dewberry is a major LIDAR production shop in the US, certainly one of the biggest.  That is a lot of staff and over the years, staff turnover is inevitable.  How do you find the learning curve for Terrasolid for new users?

Well, like most engineering software, there are many, many buttons to learn and concepts to get straight in your head.  We are processing more than 100,000 sq miles of LIDAR data this season, and though we don’t see a lot of turnover in staff, our staff has almost doubled in the past two years due to increased workload. So, we do face this issue of training our new staff, not just in Terrasolid, but also in understanding our entire production workflow. We have noticed that most new users come up-to-speed pretty quickly as we have them undergo an intensive one to two weeks of training and practice immediately after they are hired.  It’s a huge plus if the new hires are already comfortable with the MicroStation environment.  I would say a new user is productively working unsupervised after 30 days.  They won’t be using the power tools or doing the complex workflows such as developing macros, but they will be productive with the basics like doing a bare earth extraction and editing the point cloud.  And one of the hidden advantages of Terrasolid is that, unlike 10 years ago, you can find many candidates in the employment pool with significant hands-on Terrasolid experience already.

Do see an alternative or any new contenders you might want to incorporate in your production to replace Terrasolid?

Well, we do keep an eye on alternative software, and we do have other tools in our shop, which we use extensively; but for now we see no benefit to changing our workflow where we use Terrasolid.   With our investment in the suite of bare-earth extraction macros developed by our analysts for various types of data densities, sensors, vegetation, above-ground features, and terrain, as well has the new and interesting features added regularly to the Terrasolid suite, we believe that Terrasolid is reliable, robust and just works to do what we need it to do.

What’s the most interesting or unusual feature in Terrasolid you personally haven’t had a chance to use but would like to?

TerraStereo?  Viewing point clouds directly in stereo seems like it might have some interesting benefits.