A Digital Measuring and Load Planning System for Large Transport Assets

Summary. Recently, the efforts involved in the digitization and digitalization of logistics processes have grown tremendously. In line with such efforts, we investigate the potential of the process-integrated measuring and load planning of large transport assets. More precisely, considering the case of a German timber processor and retailer, we implement a digital measuring system, which performs precise measuring of regularly shaped wooden assets. The cognitive system uses laser and vision sensors, and measurements can be performed during the asset’s transportation on a forklift. The resulting data can be used to conduct a comprehensive load planning for scheduled delivery tours. The performance of our measurement system is evaluated using a small example dataset of the use case at hand. The a-priori set goal of maximum deviations of 5 cm, 7 cm and 14 cm in height, width and length, respectively, are achieved in 89% of the test cases. The proposed load planning algorithm is integrated in a commercial tour planning service to verify the feasibility of serving several customers within the same tour. We present the method’s applicability to our described use case of integrated measurement and planning.


Introduction
The digitization and digitalization of labor-intensive processes within a supply chain is an opportunity to decrease effort and time spent on non-value-added steps.One example are 2023 International Scientific Symposium on Logistics decision support systems for, e.g., load planning and routing.However, these systems typically require reliable data, which is not available in many applications.The essential role of data and data availability for the digitalization of logistics processes is also highlighted by Daxböck, Kröber, and Bergmann (2019).For work on the digitization and digitalization in general, and elaborations on the chances and risks of these processes, refer to Wei et al. (2019) and Kersten et al. (2017), for instance.
In this paper, we propose and investigate an integrated solution approach that combines a digital measurement system for measuring the dimensions of large wooden assets and a packing algorithm for truck trailer load planning.Our project partners receive and ship wooden assets from different manufacturers and with varying characteristics.While some of these assets are produced in-house and can thus be assumed to be well-known, reliable measurements of externally procured items are not regularly available.Assets are transported for customers via trucks that can transport up to 40 items with a maximum length of 18 meters and a width and height of, in most cases, no more than 2 meters.As accurate data is lacking, load planning for these routes is often performed based on the best judgement of the personnel, a process which can be error prone and leads to an increased effort for replanning and reloading.The problem arising from this use case is therefore twofold: First, a reliable system for measuring incoming assets during the process of unloading and storing assets is needed.This system should be able to measure assets while they are being moved into storage, as stopping during the transport of assets is impractical and time-consuming.Second, a fast load planning algorithm is needed that uses these measurements to check whether the assets assigned to one delivery route can be feasibly transported together and to propose a load plan for loading the trailer.Sensors and cameras are frequently used in logistics solutions (Borstell et al. 2018, Riestock et al. 2019).Camera-based systems for volume estimation exist and are, for instance, proposed by Borstell et al. (2013) and Kucuk et al. (2019).To our knowledge, our system is unique as it can measure assets exceeding the sensors' fields of view, during their transportation on a forklift.
Load planning and 3D packing algorithms have been studied for several decades (e.g., Martello et al. 2000).Problem extensions that are widely applicable are stability constraints (Ali et al. 2002), the combination of load planning with bin assignment and the integration of load planning and routing (Reil et al. 2018).In this system, load planning is tailored towards our use case (see Fig. 1) and used as a feasibility check for customer routes.

Digital Measurement
We describe the process and algorithm for the digital measuring of transported assets and evaluate the proposed method on our own specific data.First of all, the measuring environment and process are described, and necessary assumption and prerequisites are listed.Thereafter, the algorithm's essential steps are explained shortly.We introduce our use-case-specific dataset and perform evaluations of our algorithm on this data.

Measurement Process
We propose a measurement system operating in a process-integrated fashion.No additional handling or repositioning of the measurement object is required.Instead, the object is measured while being transported on a conventional forklift.The proposed system performs measurements in a designated measurement area.In the case at hand, this area is part of a warehouse corridor, above which the system's sensors are applied.
Forklifts, which are used for transporting objects during measurements, need to be labelled and registered in the system's database.Each physical forklift is labelled with a so-called Aruco marker encoding its ID.Aruco markers are 2D barcodes, which are comparably easy to recognize for computer vision systems.The marker is applied to a forklift's roof (or any horizontal surface) such that it can be observed from above.Further, the marker needs to be applied in a fixed orientation indicating the side to which the forklift carries its load.For each registered forklift, data about its geometry are stored in the system's database.This includes the height at which its marker is applied.Importantly, the bearing height of the surface on which objects are placed during transport has to be known beforehand and is fixed during transportation.
Measurements are performed one at a time: only a single forklift may enter the measurement area at a time, and no other objects of any kind may be located within the measurement area during measurements.The measuring process is triggered automatically once a forklift enters the area.The forklift must traverse the area at constant speed and movement direction for a measurement to be successful.No corners, obstacles or crossways should be located inside the measurement area or in its close vicinity.

Technical Details and Algorithm Description
We describe a digital measurement system designed to precisely determine the extents of large wooden assets during transportation on human-operated forklifts.For data acquisition, two different sensors are employed: an industrial camera obtaining 2D images from the visible and near-infrared (NIR) light spectrum, and a laser sensor obtaining 3D data points.Both devices are applied to the depot's ceiling just above a central gangway, which constitutes the designated measurement area.The devices both face vertically downward toward the ground; the application height is approximately 12 meters.Despite this height, the sensors can generally not fully capture all relevant transport assets in their respective field of view, especially when items are particularly long.Additionally, the employed solid-state Lidar (light detection and ranging) sensor produces 240,000 individual data-points per second rather than discretized recordings of the sensor's whole field of view.For these reasons, measurements are performed over a duration of multiple seconds, requiring the temporal dimension to be taken into account.
Our measuring algorithm can be subdivided into five steps, which we briefly summarize.The first step concerns the automated start of measurements.The measurement process is started when the system detects an Aruco marker applied to a forklift.Once a measurement is started, 2023 International Scientific Symposium on Logistics the system's second step is to analyze the camera data and to determine the relevant timeframe of the measurement.The camera images are also used to determine the forklift's trajectory through the measurement area and deduce the movement speed and direction in real-world coordinates.In a third step, the 3D Lidar data of the measurement is extracted.As a measurement is performed over a time interval of multiple seconds, each data point is spatially corrected corresponding to its individual timestamp.Assuming precise measurements and trajectory information, this results in a movement-corrected, consistent 3D point-cloud depicting forklift and measurement object.In step 4 of our algorithm, the 3D point-cloud is clustered using the algorithm DBSCAN (Ester et al. 1996).The goal of the clustering is to identify all the 3D data points, which depict part of the logistics asset to be measured, as opposed to such points depicting parts of the transportation device, i.e., the forklift.We exploit given knowledge about forklift architecture, orientation and direction of movement to pick the correct cluster to be measured.In a fifth and last step, the asset size is determined based on the selected cluster's extent.The cuboid-shaped asset is assumed to be aligned to the floor, thus, object height and object length and width are determined independently.

Evaluation
Data acquisition for the evaluation of our digital measuring system requires high manual efforts and availability of adequate resources.As such are limited, we evaluate our algorithm on a small dataset of 36 instances.Three different forklifts were used in data acquisition: A conventional front-loading forklift, a side-loading forklift, and a so-called four-way forklift.The latter can freely move in any horizontal direction.Each data instance consists of a series of 2D images, with respective timestamps, and the set of 4-dimensional laser sensor data points (x, y, z coordinates and timestamp).Additionally, each instance is annotated with the ground-truth measurements of the object (length, width, height), which were manually recorded.The type of transporting forklift and its geometry is also available for each data instance.
We perform digital measurement of all data instances as described above and record absolute and relative deviation of measured length, width and height.The corresponding quantitative results for all 36 instances are listed in Table 1.The average deviations of measured size compared to real size are 6.8 cm in length, 3.9 cm in width and 2.0 cm in height.Average relative deviations read 2.5%, 2.9% and 5.7% in length, width and height, respectively.

Average Deviations
Instances In coordination with our industry partner, we defined deviation thresholds which are to be achieved to classify a measurement as accurate.These thresholds are set to 5 cm in height, 7 cm in width and 14 cm in length.We report the accuracy of our measurement algorithm based on these defined deviation thresholds on our test data in the last row of Table 1.Overall, 32 of our 36 instances were measured correctly, corresponding to an accuracy value of 88.89%.

Load Planning
The detailed measurements of the wooden assets are needed to determine, in a second step, feasible load plans for delivering these assets to different customers.To this end, a Constraint-Programming-based algorithm for load planning is implemented that accounts for several additional requirements relevant for our use case: Trailers can be accessed from both sides along their longitudinal axis.This is represented by modelling the trailer as having two bins into which items can be placed, accessed from the right and from the left hand side of the truck, respectively.Items may exceed the trailer in length, provided that their center of gravity lies within the limits of the trailer itself.Load planning should furthermore account for a predefined order of customers such that items can be unloaded at intermediate stops without additional handling operations.The load planning system is combined with an existing, commercial solver for determining the sequence of customer stops so as to evaluate whether all assets assigned to one customer tour can be feasibly transported together.
A Constraint Programming (CP) model consists of a set of variables, variable domains and constraints that are defined as follows in our use case.For modelling the load planning problems, decision variables indicate whether items are packed on the right hand side or on the left hand side of the truck.The position of the bottom left corner of the assets determines their position in the 3D loading space.Note that, due to the regular shape of the items, we do not account for item rotation.Constraints ensure that all items are placed inside the bounding box of the bin they are assigned to.If they exceed the trailer along the third axis, these constraints moreover ensure that their center of gravity lies within the defined bounding box so as to ensure stability.
We apply the global no-overlap constraint provided by standard CP solvers to ensure that all packed items are disjoint in space.The constraint is an abstract formulation that, when implemented by CP solvers, ensures that the rectangular shapes defined by the size vectors do not overlap with one another.Note that while the constraint could be replaced by a series of constraints ensuring that all possible pairs of items do not overlap with one another, stating the restriction as a global constraint allows for the solver to use a more efficient algorithm for ensuring feasibility and eliminating infeasible candidate positions.Finally, an auxiliary objective minimizes the sum of position values to ensure that items are placed as closely towards the bottom front edge of the trailer as possible.
The model formulated above is implemented in C++ and solved using the Gecode CP solver (see Schulte et al. 2006).Note that, even though the model is formulated as an optimization model, its primary function is to serve as a feasibility check with a short computation time instead of determining the overall optimal solution.In other words, while the objective function indicates a preference for beneficial arrangements of items within the trailer, we interrupt search early if a feasible solution is found.

Summary and Outlook
We presented a solution for the process-integrated measurement of large wooden assets and subsequent load planning.A NIR camera and a solid-state laser sensor are used to obtain measurable 3D-data of an asset during its transportation on a conventional forklift.Notably, the system is designed to measure large objects exceeding the sensors' fields of view, as measurements are performed over an adequate time period.The system achieves the pre-defined required accuracy in approximately 89% of test cases.We propose a Constraint-Programming-based load planning algorithm to produce feasible and optimized loading plans for the measured wooden assets.Combined with the measurement system and a commercial routing engine, this ensures that tours for delivering assets to customers can be determined such that assets can be feasibly loaded into the trailer based on accurate measurements.
Multiple directions for further research on the topic exist.Regarding the measurement system, a possible enhancement could be its application to other domains handling non-wooden assets of arbitrary sizes.The extension of the system to allow for the simultaneous measurement of multiple objects on a single forklift is another promising prospective.To achieve a robust solution in this case, the further employment of the camera's visual data in image analysis methods can be beneficial.To allow for even more sophisticated tour and load plans, integrating the proposed load planning system into existing tour scheduling processes instead of performing a post-planning feasibility check could be advisable.For non-wooden and potentially smaller assets, the packing algorithm should moreover be enhanced by the consideration of item rotation and more sophisticated search steps that allow for the efficient planning of a larger number of items.

Figure 1 .
Figure 1.Visualization of truck load planning.Items may exceed the length of the trailer within a limited extend (yellow box) and can be accessed from the left and from the right hand side, indicated by the two parallel red boxes.