GROUND-BASED REMOTE SENSING WITH LONG LENS VIDEO CAMERA FOR UPPER-STEM DIAMETER AND OTHER TREE CROWN MEASUREMENTS
Neil Clark Research Forester, SRS 4702 Forest Products Conservation 1650 Ramble Road, Blacksburg, Virginia 24060 (540)231-4674 (ph) (540)231-1383 (fax) nclark@fs.fed.us Sang-Mook Lee Researcher, Bradley Department of Electrical Engineering Virginia Tech, Blacksburg, Virginia 24061 lsmook@vt.edu

ABSTRACT

This paper demonstrates how a digital video camera with a long lens can be used with pulse laser ranging in order to collect very large-scale tree crown measurements. The long focal length of the camera lens provides the magnification required for precise viewing of distant points with the trade-off of spatial coverage. Multiple video frames are mosaicked into a single super-resolution image to increase the spatial coverage. Upper-stem diameters are the examples given here, but the technique may be generalized to other dimensions of interest.

INTRODUCTION

One of the long-term limitations for tree stem measurement has been the inability to non-destructively collect dimensional measurements of anything above diameter at breast height (DBH). Most sampling applications utilize DBH and maybe an optically determined height measurement as input variables to a model to determine the dependent variable of interest. These models were typically created using destructive sampling. The ability to obtain a model created from a sufficient number of observations of an appropriately selected population may be difficult. Sometimes the need arises for non-destructive sampling of a more specific population. Optical dendrometers have been created for this purpose for data needs where existing models based on DBH are not available (Grosenbaugh 1963). The problem presented in this paper is generally stated as the need to measure a large number of inaccessible objects (e.g., upper stem diameters or individual tree crown measurements). A digital solution composed of a long lens video camera and pulse laser ranging used manually from the ground is proposed.

TMS INSTRUMENT

A prototype instrument for the Tree Measurement System (TMS) was created by Laser Atlanta, Inc.*, which is their Advantage CIL laser rangefinder modified with a digital video camera and custom lens system (Clark 2000, Clark et al. 2001). The laser rangefinder outputs 10 records per second (subsampled from 238 readings per second) for range and triple-axis orientation (RO). The video camera is a Panasonic GP-CX161 charge coupled device (CCD) chip that outputs an NTSC standard video signal. Currently, this signal is recorded to a Sony GV-D300 portable video cassette recorder for storage. At the time of most of these studies the signal was digitized using a frame-grabber. Recently this has been improved to use the DV format. In both cases the final captured frames were 480(v) x 720(h) pixels. The field of view (FOV) of the lens system is 5.5 degrees high by 8.24 degrees wide. In
The use of trade, firm, or corporation names in this publication is for the information and convenience of the reader. Such use does not constitute an official endorsement or approval by the U.S. Department of Agriculture of any product or service to the exclusion of others that may be suitable. Remote Sensing for Field Users Proceedings of the Tenth Forest Service Remote Sensing Applications Conference Salt Lake City, Utah April 5-9 2004
terms of 35mm film camera equivalency this would be a 250mm lens. This results in an image resolution of 0.2 mm per meter of object space distance.
Figure 1. TMS instrument on monopod with portable VCR.

PROCESSING

Unfortunately the attainment of a measurement using this technique is more complex than many destructive methods or heuristically-based optical devices. Since digital devices are inherently quantitative, a certain amount of processing is required to obtain the information in a format meaningful to the user. Some of this need for processing is due to the design of hardware elements that are currently available. Also, additional processing is required for better results. Synchronization, mosaicking, super-resolution, segmentation, and model corrections are some of the processing steps that must occur to extract the variables that are meaningful to the user. As in other types of remote sensing applications the FOV is too small to capture the entity being measured in one frame. In aerial applications, typically the FOV is limited by flying height and spatial resolution specifications to enable the examination of individual tree crowns, but the spatial extent may be a stand or entire region. Using the TMS instrument the distance from the stem is limited by visibility considerations, sufficient spatial resolution is at the Nyquist frequency of the measurement precision required, and the spatial extent is typically an entire tree stem. In both the aerial and TMS cases, many image frames must be mosaicked. Aerial photo analysis uses analytical methods or sophisticated equipment (i.e. stereoplotter) to temporarily orient the image data so that the measurement information can be extracted. More modern seamless methods allow the image data to be processed on the fly, so that terrain products (e.g., digital elevation models) and corrected image products (e.g., digital orthogonal photography) are created very rapidly. The TMS also uses a similar streamlined process to output information. The TMS has a PC-based computer program that automates much of these processing tasks. Due to the use of two commercial-off-the-shelf (COTS) hardware pieces, the image and the RO data are output to two separate streams. These two data streams need to be synchronized in order to be useful. This task is not too difficult as both data streams share a common starting point and are recorded in regular time intervals. The RO data files are named sequentially and the video data should have been recorded to tape in the same sequential order. When the user runs the TMS program back at the office the RO data is used to guide the frame-grabbing function and the data is synchronized. Remote Sensing for Field Users Proceedings of the Tenth Forest Service Remote Sensing Applications Conference Salt Lake City, Utah April 5-9 2004

This synchronization step aids the next processing phase -- the mosaicking procedure. The mosaicking algorithm uses this range and orientation information as parameters to limit the search space, improve efficiency, and provide a discriminating factor in cases of ambiguous matches. The current mosaicking method uses Gaussian image pyramids in a coarse-to-fine scheme to refine the mosaic. Kee et al. (2002) found the normalized sum of absolute deviations to be sufficiently accurate and the most efficient for determining match strength. Mosaicking allows measurements to be obtained for scale spaces larger than a single image frame (e.g., tree height or crown measurements). Figure 2 shows a typical mosaic of the main bole of a stem. The stair-step edge indicates the small segments which were obtained from individual video frames. The compromise of digital video compared to digital still image is the substitution of temporal resolution for spatial resolution (Table 1). Though some larger video chips are becoming available, they have understandably been limited by display standards. Now that digital display is becoming more mainstream, the spatial resolution of video chips is following. Super-resolution has been proposed as a way to increase spatial resolution by registration of multiple video frames containing small amounts of motion between frames (Huang and Tsai 1984, Baker and Kanade 2002). At this point only two captured frames are being used for super-resolution (Kee 2003); however future work will examine potential benefits of using more video data. Table 1. Comparison of video versus still imaging Video Storage cost Spatial resolution Temporal resolution Iris Lens $3US / 112Gb 480 x 720 per frame Interleaved 30 frames / second Auto gain control cheaper per unit area Limited commercial availability of fixed lenses $200US / 1 Gb 3000 x 4000 instantaneous 3 frame burst mechanical / many options Greater range Commercially available Still
Finally, image segmentation and edge-detection is essential for the automated extraction of stem diameters. Image segmentation is challenging in the cluttered, sub-canopy environment where multiple layers of overlapping objects frequently occur. These objects may at times be represented by very similar spectral properties. As this is the case, the instrument motion field, range discrimination, and stem form constraints are all used in order to aid in locating the edges of interest. The red crosshairs in figure 3 indicate the motion vector of the RO points of the TMS instrument as the stem is scanned. The edge finding algorithm detects strong edges emanating from these points as depicted by the yellow outlines. Overlapping structures cause edges to be found at incorrect locations. Stem form constraints are then used to refine the final edge detection. More work is still being done to improve this procedure, but this general description is presented here for demonstration purposes.

Remote Sensing for Field Users Proceedings of the Tenth Forest Service Remote Sensing Applications Conference Salt Lake City, Utah April 5-9 2004
Figure 2. On the left is the rough mosaic generated from the raw RO data from the TMS instrument. The right portion of the image shows the results after the mosaicking procedure.
Figure 3. Depiction of the image segmentation procedure on a stem component with overlapping structures. The red crosshairs represent the motion vector of the RO point of the instrument. The yellow outline represents the first strong edges found emanating from the RO points. The green area shows edges constrained by stem form parameters.

CORRECTIONS

Beyond the processing steps mentioned above a number of corrections are applied to increase the accuracy of the overall model. The object shape and perspective model is applied in cases where stem diameters are being measured. This applies to crown diameters as well. The range information that is being used to scale the image space measurement is located in front of the points on the stem that represent the diameter. As trees tend to have a circular cross section, this is used as the model. If multiple viewpoints are available for the tree an elliptical cross section can be applied. Corrections are also made based on perspective. If available, a portion of the range information is used to determine the stem axis orientation relative to the camera axis. This adjusts the scale and also the location of the points representing the stem edge. As the angle between then stem axis and camera axis becomes too acute, the data is of limited usefulness as the correction gets exponential and there is a great chance of foreground occlusion by other parts of the stem. For instance, it is very difficult to measure a branch that is pointing directly at the camera. Range measurements can not be obtained with confidence and the location along the length of the branch that the edges are perceived is questionable. Unlike an aerial application where the image plane is being scanned at a near parallel path to the imaged object (the ground). Here the TMS instrument is rotated from a fixed location. The motion parameters must be properly accounted for in order to create an appropriate model. To this point an assumption has been made that the rotation of the camera is about the focal point. This has not been tested however. As the camera is typically used with a Remote Sensing for Field Users Proceedings of the Tenth Forest Service Remote Sensing Applications Conference Salt Lake City, Utah April 5-9 2004

monopod, the points of rotation are fixed. However, corrections must be made for any movement of the monopod during camera rotation. Correction for each scanning session is not practical; however testing should be done to determine the severity of this movement and its effect on overall error. Improvements can be made by determining the most frequent rotation point and trying to limit movement while scanning the stem. A scanning mirror could be used to remedy moving the entire unit; however this would add considerable bulk and fragility to the system.

APPLICATIONS

The TMS may be used to measure upper stem diameters, bole form, taper, defects, and crown architecture components such as branch metrics and profile models for precise surface area or crown volume estimation (Clark 2001). Work is also being done for foliage assessment such as crown density and crown transparency measurements (Lee et al. in press), which are currently part of the forest inventory and analysis (FIA) Phase 3 collection effort (USDA-FS 2002). Clark et al. (2001) compared TMS instrument measurements with pentaprism caliper estimates and mechanical caliper measurements. These results were not very promising as very large errors were present for both the TMS instrument and the pentaprism estimates. Subsequently, a few errors were discovered in the camera method calculations. Lack of directional and height controls were suspected to have caused the large errors, as they were evident even among the traditional methods. This lead to further study comparing the TMS instrument with a Barr & Stroud FP-12 optical dendrometer on diameters at randomly selected, marked heights. The Barr & Stroud dendrometer possesses excellent optics and has been shown to be extremely accurate (Bell and Groman 1971). These results are not yet published but look more promising. Another study is underway using marked diameters and felled tree measurements.

ERRORS

Errors may occur at many places within this process. The first errors that must be realized and specified are the ranging and imaging measurement errors. The reported error of the laser-ranging instrument is 15 centimeters. This would be more significant at shorter object space distances, as it would be a higher proportion of the overall object distance. This reported error also discounts additional error that may occur due to target confusion as the laser may strike other crown components or understory elements. There are filters in place to prevent severe errors of caused by this occlusion, however a more subtle discrepancy may go unnoticed. Image distance measurement precision is 0.2 mm per meter of object space distance, so image distance error is more significant at larger object space distances. This is compounded by perspective effects at acute viewing angles and the more frequent observation of smaller dimensions at larger distances. Though both object space and image space errors can be quite significant, object space distance measurement is more difficult to account for after the fact.

FACTORS AND EFFECTS

There are a multitude of other factors and effects that contribute to measurement variability or the ability to even collect any information. The primary reason to use remote sensing methods is due to lack of access. This is still a problem given the dense, gordian nature of a tree crown or dense forest condition. There are many occluding structures that create problems. Multiple viewpoints can be used to mitigate some of these problems, but depending on density and orientation access to some portions of the crown, or particularly foliage, will always be difficult. Light reflectance is essential for passive remote sensing in the visible portion of the spectrum. For many measurement applications light is controlled. Absolute light control is not possible for practical forest sampling as sunlight is the default light source. Sampling at night using artificial light is a possibility, but rarely practical. Limitations may be placed on the time of day, day of year, generalized atmospheric conditions, or orientation of light to imaged object. Contrast, specular reflectance, and other edge errors are significant impediments to accuracy and repeatability. These factors are very difficult to control in practical monitoring situations. Contrast extremities can be present among the pixels representing the stem edge. If the stem edge pixel is adjacent to a bright sky pixel the light, the Remote Sensing for Field Users Proceedings of the Tenth Forest Service Remote Sensing Applications Conference Salt Lake City, Utah April 5-9 2004
blooming effect may shift the edge towards the tree center. Another branch or tree directly behind the edge may limit the contrast shifting the edge away from the tree center. Model errors also contribute to measurement variation. The interior camera orientation model can have an effect. This can be corrected for providing that the hardware elements remain stable. Digital processing manipulations including sampling, white balancing, auto gain control, etc. are all modeled factors that may affect image space measurement. External model factors include the camera motion model and the stem shape and orientation.

SUMMARY

The instrument described in this paper combines imaging and ranging to collect ground-based data on individual tree stems. The small FOV allows for the needed measurement precision, while the instrument orientation information and image processing components provide for information extraction over a much larger spatial region. Using a highly magnified digital imagery in combination with laser ranging can provide a suitable solution for a number of individual tree stem sampling applications such as upper stem diameters, bole form, taper, defects, and foliage and crown architecture components. This paper presents one such prototype instrument. Improvements can be made by using better cameras or lens systems. The difficulty that still remains for many upper-stem assessments is access. Sensor location near the ground and the gordian nature of tree crowns limit the visibility of structures that require measurement.

FUTURE WORK

Recent hardware developments are occurring that will enable the creation of a field-ready solution without the need for large amounts of custom engineering. IEEE 1394 and USB2.0 high performance serial bus cameras are now commonly available allowing them to easily be connected to field-ready computers. This improvement will eliminate the need for a VCR and enable real-time measurement. Getting all of the data streams synchronized, transferred, and processed to a desired state in the field is quite desirable for practical use. This would also be coupled with data reduction strategies in order to store only needed information. Alternate cameras or lens systems may be explored based on the desired information requirements. A lens system with greater magnification, or a line-scan laser or camera could be used for single upper-stem diameters. Use of the range map from current instrument to augment other imaging device could be an alternative low-cost solution. Further work will be done to incorporate better camera, motion, and object models. Multiple view angles for 3D reconstruction are also being considered. Remeasurement studies will also be done determine reliability for varying applications.

REFERENCES

Baker, S., and T. Kanade. 2002. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence. 24(9):1167-1183. Bell, J.F., and W.A. Groman. 1971. A field test of the accuracy of the Barr and Stroud Type FP-12 optical dendrometer. The Forestry Chronicle. 47(2):6974. Clark, N. 2000. Initial results from a video-laser rangefinder device. Proceedings, SOFOR GIS 2000, 3rd Southern Forestry GIS Conference. D1: 6 pp. Clark. 2001. New instrument expanding individual tree stem analysis. Proceedings of the Southern Forest Science Conference. Atlanta, Georgia. Nov. 26-28, 2001. 6 pp. Clark, N., S. Zarnoch, A. Clark III, and G. Reams. 2001. Comparison of standing volume estimates using optical dendrometers. In Proceedings of the second annual Forest Inventory and Analysis symposium; Reams, Gregory A.; McRoberts, Ronald E.; Van Deusen, Paul C., eds.; 2000 October 17-18; Salt Lake City, UT. Gen. Tech. Rep. SRS47. Asheville, NC: U.S. Department of Agriculture, Forest Service, Southern Research Station. 123-128. Grosenbaugh, L. 1963. Optical dendrometers for out-of-reach diameters: A conspectus and some new theory. Forest Science Monograph. #4. 47 p. Remote Sensing for Field Users Proceedings of the Tenth Forest Service Remote Sensing Applications Conference Salt Lake City, Utah April 5-9 2004
Huang, T., and R. Tsai. 1984. Multi-frame image restoration and registration. Advances in Computer Vision and Image Processing. 1:317-339. Kee, E., Clark, N., and A.Abbott. 2002. Automated mosaicking of sub-caonopy video incorporating ancillary data. Proceedings of Symposium on Statistics and Information Technology in Forestry. Sept 8-12, 2002. Blacksburg, Virginia, USA. p. 36-44. Kee, E. 2003. Video mosaicking using ancillary data to facilitate size estimation. Masters Thesis. Virginia Tech, Blacksburg, Virginia. 134 p. http://scholar.lib.vt.edu/theses/available/etd-05132003-221519/ (7 May 2004). Lee, S., Clark, N., and P. Araman. In Press. Automated methods of tree boundary extraction and foliage transparency estimation from digital imagery. Proceedings of the 19th Biennial Workshop on Color Photography & Videography in Resource Assessment. October 6-8, 2003. Logan, Utah, USA USDA-FS. 2002. Phase 3 Field Guide Crowns: Measurements and Sampling. March 2002. http://fia.fs.fed.us/manuals/p3sec12_02-03_final.doc (7 May 2004).