Comparative Analysis of UAV Platforms for Coastal Monitoring Applications
Evaluating consistency during the transition between DJI Phantom 4 RTK and DJI Mavic 3 Multispectral
Abstract
This research evaluates the consistency and performance between two distinct UAV platforms—the DJI Phantom 4 RTK and the DJI Mavic 3 Multispectral—in generating photogrammetric products for coastal terrain analysis. Through rigorous comparative analysis of point clouds, Digital Surface Models (DSMs), Digital Terrain Models (DTMs), and orthomosaics, this study provides critical insights into the transition between these platforms for ongoing coastal monitoring programs.
Key Findings
- Both platforms demonstrated comparable accuracy with horizontal RMSE of 3.0cm and vertical RMSE of 3.5cm
- The Mavic 3M offers additional multispectral capabilities while maintaining RTK positioning accuracy
- Processing efficiency improved with the Mavic 3M, requiring 30% less processing time
- The transition to Mavic 3M is justified, offering comparable accuracy with enhanced capabilities
Methodology
Data Acquisition
The study compared a legacy dataset acquired using the DJI Phantom 4 RTK in 2022 with a newly acquired dataset from the DJI Mavic 3M. Both UAV datasets were collected over the same geographical area at the Queensland Government Hydraulics Laboratory (QGHL) site in Deagan, Brisbane, to ensure a controlled comparison of photogrammetric outputs.
| Flight Parameter | Phantom 4 RTK | Mavic 3M |
|---|---|---|
| Camera | FC6310R_8.8_5472x3648 (RGB) | M3M_12.3_5280x3956 (RGB) |
| Target GSD (cm/pixel) | 1.38 | 2.15 |
| Flight Altitude (m) | 80 | 80 |
| Side Overlap (%) | 70 | 80 |
| Front Overlap (%) | 70 | 80 |
| Flight Path | Crosshatch | Grid |
Table 1: Summary of flight parameters for both datasets
Data Processing and Analysis
Both UAV datasets were processed using Pix4D Mapper to generate orthomosaics, point clouds, DSMs, and DTMs. A change detection analysis was performed to identify areas with significant temporal variations, and sample areas were selected from locations with minimal temporal variations for detailed comparison.
Figure 1: Map overlay showing the selected stable features used for comparison at the QGHL site
Each sample area underwent statistical outlier removal and was analyzed using a comprehensive set of metrics including vertical and horizontal displacement, point cloud characteristics, absolute accuracy metrics, and data distribution metrics.
Key Results
Displacement Measurements
The comparison between the two UAV systems across three sample areas revealed consistent accuracy with progressive increases in displacement from raw data (point clouds) to processed outputs (DTMs), illustrating the cumulative effects of error propagation through the processing workflow.
| File Type | Displacement Type | Overall RMSE (m) |
|---|---|---|
| Point Cloud | Horizontal | 0.030 |
| Vertical | 0.035 | |
| Total 3D | 0.046 | |
| DSM | Horizontal | 0.018 |
| Vertical | 0.032 | |
| Total 3D | 0.038 | |
| DTM | Horizontal | 0.093 |
| Vertical | 0.426 | |
| Total 3D | 0.338 |
Table 2: Summary of RMSE displacements between the P4 RTK and M3M
Point Cloud Comparison
Point cloud comparison revealed generally consistent results between platforms, with horizontal displacements averaging 0.03m and vertical displacements of 0.035m across all samples. While the P4 generated denser point clouds (up to 1199.8 points/m²), this was primarily attributed to the crosshatch flight pattern rather than inherent system capabilities.
Figure 2: Box and Whisker plots showing horizontal, vertical and total displacement between P4 and M3M point clouds across three sample areas
Orthomosaic Comparison
Visual inspection and overlay analysis of the orthomosaics generated from both UAV systems revealed highly consistent results. The only notable differences were camera viewing angles and variations in lighting conditions due to different capture times. No geometric distortions or warping effects were observed in either dataset.
Processing Efficiency
The Mavic 3M demonstrated improved processing efficiency, requiring 13m:07s compared to the Phantom 4 RTK's 18m:39s. The M3M also achieved better image calibration (100% vs 99%) and higher median matches per image (16090.6 vs 8927.21).
| Metric | Phantom 4 (P4) | Mavic 3M (M3M) |
|---|---|---|
| Processing Time | 18m:39s | 13m:07s |
| Images Calibrated | 252 out of 253 (99%) | 160 out of 160 (100%) |
| Camera Optimization | 0.73% | 1.44% |
| Median Matches per Image | 8927.21 | 16090.6 |
| Mean Reprojection Error | 0.194 pixels | 0.213 pixels |
| Mean RMS error for GCPs | 0.011 m | 0.008 m |
Table 3: Comparison of key processing metrics between platforms
Discussion
The comparative analysis between the DJI Phantom 4 RTK and Mavic 3M demonstrated both systems' capabilities in generating high-quality photogrammetric products for coastal monitoring. The assessment focused on point clouds, DSMs, DTMs, and orthomosaics across three sample areas, revealing insights into system performance and operational considerations.
Accuracy Comparison
The achieved accuracies align well with established benchmarks from recent coastal monitoring literature. The overall RMSE values (horizontal: CE90 0.033m, vertical: LE90 0.051m) fall within acceptable ranges for coastal monitoring applications, particularly considering established benchmarks of 3.7cm horizontal and 5.3cm vertical accuracy.
Platform Advantages
The M3M offers several advantages over its predecessor, including additional multispectral capabilities (NIR and Red Edge bands) while maintaining comparable RTK positioning accuracy. Processing metrics showed improved efficiency with the M3M, requiring a simpler flight path to produce similarly accurate results. The consistent point cloud density ratios (around 0.37) suggest fundamental improvements in the M3M onboard software.
Limitations
The study's primary limitation was its reliance on a single comparative test. A more robust analysis incorporating multiple flights under various conditions and locations would provide better statistical validation. Additionally, a reference LiDAR dataset would be superior to this experiment's direct comparisons.
Conclusion
The results demonstrate that both platforms meet the accuracy requirements for coastal monitoring applications, with differences falling within expected operational error ranges. The transition from P4 to M3M appears justified, offering comparable accuracy with additional spectral capabilities and improved processing efficiency.
The DJI Mavic 3M performs comparably to the Phantom 4 RTK, achieving horizontal RMSE accuracies of 3.0cm and vertical RMSE accuracies of 3.5cm, while offering additional multispectral capabilities and improved processing efficiency. These results validate the platform transition while maintaining the high accuracy standards required for coastal monitoring.
Future work should focus on leveraging the M3M's enhanced multispectral capabilities while maintaining the demonstrated level of geometric accuracy.
Related Research
Contact Us
Interested in discussing this research or exploring similar projects? Our team of experts is ready to help.
Get in Touch