Assessment of Temperature Effects in Interior Orientation Parameters Calibration of Optoelectronic Devices

A digital micromirror device (DMD) micromirrors periodic spatial structure is a measuring scale in interior orientation parameters calibration of optoelectronic devices problems, when using a DMD as a testobject. It is important that DMD micromirrors periodic spatial structure remains constant. Change in a DMD micromirrors spatial structure may occur due to heating. In addition to heating a DMD, an optoelectronic device photodetector is also subject to heating and, accordingly, change in its spatial structure. It is necessary to estimate change in a spatial structure of DMD micromirrors and an optoelectronic device photodetector. A DMD micromirrors spatial drift and a DMD micromirrors spatial drift together with a digital camera photodetector pixels spatial drift for operation 4 h are analyzed. The drift analysis consisted in the points array position assessing formed by a DMD and projected onto a digital camera. When analyzing only a DMD micromirrors drift, a digital camera was turned on only for shooting time for exclude digital camera influence. A digital camera did not have time to significantly heat up, during this time. After a digital camera it cooled to a room temperature. Average drift of all DMD micromirrors determines the accuracy of interior orientation parameters calibration of optoelectronic devices using a DMD in time. Maximum drift of all micromirrors after switching on is observed. Minimum DMD warm-up time is 60 min for average drift of all micromirrors less than 1 μm is necessary. Minimum DMD warm-up time is 120 min when using a DMD together with a digital camera is necessary. A DMD expansion uniformity determines the accuracy of interior orientation parameters calibration of optoelectronic devices using a DMD, because irregular expansion disturbs micromirrors periodicity. The average change in distance of neighboring points is less than 0.1 μm for every 20 min. Thus, a DMD can be used as a test-object in interior orientation parameters calibration of optoelectronic devices. The results can be used as compensation coefficients of change in DMD micromirrors spatial structure due to temperature effects during operation, if more accurate are necessary.

Ключевые слова: геометрическая калибровка, оптико-электронный аппарат, температурный дрейф, фотоприёмник, цифровое микрозеркальное устройство. DOI: 10.21122/2220-9506-2020-11-2-122-131 Introduction When using a digital micromirror device (Digital Micromirror Device -DMD) as a testobject, the micromirror sizes of which are a measuring scale, in particular in the problems of interior orientation parameters calibration of optoelectronic devices [1][2] it is important that the spatial structure of micromirrors remains constant. The spatial structure of micromirrors may be disturbed due to heating during operation and, accordingly, expansion of the CMOS matrix, in the cells of which micromirrors are installed [3]. The change in the spatial structure of micromirrors can be compensated. With uniform expansion, this can be done with a single scale factor or a scale factor horizontally and vertically. It is more difficult to compensate for uneven expansion; this requires a polynomial for each DMD section, depending on the duration of the DMD operation. Shifting all DMD micromirrors in one direction affects interior orientation parameters over time. Thus, to assess the possibility of using DMD as a device, which is a testobject for interior orientation parameters calibration of optoelectronic devices, it is necessary to evaluate the change in the spatial structure of DMD depending on the duration of operation, as well as the possibility of compensation. A change in the spatial structure of less than 1 μm is acceptable for most tasks of interior orientation parameters calibration of optoelectronic devices and is comparable in accuracy with highprecision two-coordinate stages [4] in which you can install a test-object in the form of a glass plate with a pattern deposited on it with a photolithographic method and use it for interior orientation parameters calibration of optoelectronic devices.
In problems of interior orientation parameters calibration of optoelectronic devices, the matrix structure of the photodetector is also a measuring scale [5]. In addition to heating the DMD, the optoelectronic device photodetector is also subject to heating, and, accordingly, to a change in its spatial structure. The purpose of the investigations was the need to estimate the value of the change in spatial structure of the photodetector calibrated by the optoelectronic device so as not to set excessive requirements on the accuracy of measuring the distortion of the lens, since its value may exceed the change in the spatial structure of the photodetector during the calibration time and the operation of the optoelectronic device as intended. This assessment will make it possible to decide on the duration of operation during which a change in the spatial structure of the optoelectronic device photodetector is acceptable, as well as on the need for an additional cooling system for the optoelectronic device photodetector.

Description of scheme
Calibration consists in projecting a test array consisting of a matrix of points with a constant distance between points on the camera photodetector and assessing the change in the distance between points over a long time of DMD operation, which will change due to heating of the DMD. The DMD calibration pattern is similar to interior orientation parameters calibration of optoelectronic devices [6,7]. In this case, the DMD lens-camera lens system magnify the DMD image 3-5 times to increase all DMD manufacturing errors, whereas in to interior orientation parameters calibration of optoelectronic devices schemes it is customary to use the collimator-optoelectronic device lens system that reduces the image 3-1 times to reduce the effect of manufacturing errors: alignment, calibration of the collimator.
The realizeed calibration scheme, block diagram of which is shown in Figure 1, and calibration test set-up in Figure 2, includes the DLP LightCrafter 4500, f ' = 500 mm lens, which projects the DLP LightCrafter 4500 image into the collimator. DLP LightCrafter 4500 includes a DMD that is illuminated by an illumination system. In the DLP LightCrafter 4500, whose appearance during operation is shown in Figure 3, the full-time lens is removed. Instead of a full-time test-object, a Basler ace acA2040-90 um (CMOS photodetector CMOSIS CMV4000) [8] camera is installed in the collimator. The collimator housing is made of invar and has a low coefficient of linear expansion, and the f ' = 1500 mm collimator lens is designed and constructed in such a way as to ensure minimum thermal stability, i. e. the mounting materials of the optical components compensate for the effect of temperature changes on the optical components, which ensures the stability of optical characteristics. This ensures stability in the first place, the position of the focal plane of the collimator lens f ' = 1500 mm relative to the plane of the camera photodetector.
DMD is a bistable spatial light modulator, including an array of rotating micromirrors functionally installed in memory cells on a CMOS matrix. The CMOS matrix controls the position of the micromirrors. It is possible to control the position of the micromirrors using software and thus form the desired pattern, which can be used as a test-object. Since micromirrors are installed in the cells of the CMOS matrix, the DMD is characterized by thermal effects like cameras with CMOS photodetectors.   To eliminate distortions in the optical path of the calibration scheme due to air refraction, the optical path is closed by the cover. The use of the cover removes the influence of the movement of air masses in the room on the calibration process. The cover is installed so that the calibration schemes of the illuminator and camera parts that are not included in the optical path remain open. This is necessary to freely remove the heat generated by them during calibration. In addition, the cover must be porous to ensure heat removal from the optical path, as well as to eliminate stagnation of air masses. The room temperature requirements [5] are the same as for interior orientation parameters calibration of optoelectronic devices 1 . Changes in temperature in the room are minimized (during the measurement period they were no more than 0.1 °С).
To reduce the effect of vibrations, the calibration scheme is mounted on a vibration isolation bench. The described methods for removes the influence of external factors make it possible to bring the calibration closer to ideal conditions and directly measure the temperature drift.

Description of methodology
Calibration of DMD 1. DMD, lens f ' = 500 mm, collimator are visually aligned.
2. Camera and DLP LightCrafter 4500 turns on. Pattern of the test-object with a required period and the size of points is loaded into DMD.
3. Illuminator current value of DLP LightCrafter 4500 or camera exposure is determined, to provide illumination of the camera photodetector at the level of 80-90 % of the value of its saturation capacity. Several layers of astrolon are installed in the illumination system when the adjustment range of the illuminator 1 GOST 8.050-73 State system for ensuring the uniformity of measurements. Normal conditions for linear and angular measurements (with Change No. 1) Devices andMethods of Measurements 2020, vol. 11, no. 2, pp. 122-131 M.A. Starasotnikau Приборы и методы измерений 2020. -Т. 11, № 2. -С. 122-131 M.A. Starasotnikau current is insufficient, which works like frosted glass reduces luminous flux by about 50 % and makes it uniform. The frequency of micromirrors operation is selected together with the illuminator current value so that, due to an synchronization error of the operation of individual micromirrors or micromirrors blocks of the DMD and because of inconsistency in the frequency of operation of the DMD and the camera, flicker is not observed. It is checked that the test-object pattern is not formed by non-working micromirrors (micromirrors that do not change position from the nominal state to the working state, for DLP LightCrafter 4500 no more than 10 pieces on DMD), if necessary, a test-object pattern shifted by several micromirrors is loaded in the DMD. 4. The image from the camera's photodetector achieves the location of the DMD pattern in the center of the focal plane by turning the collimator around three axes. 25 points of the DMD pattern are projected onto the camera.
5. Camera and DLP LightCrafter 4500 turn off and cool to room temperature. 6. DLP LightCrafter 4500 turns on. 7. Camera turns on. 100 frames of the DMD pattern are shot at a frequency of 90 frames/s. After shooting, the camera immediately turns off, i. e. power is removed from it. This allows us to exclude the influence of the temperature drift of the camera when analyzing the temperature drift of DMD, since the camera does not manage to heat up significantly during this time -less than 1 min. The camera cools to room temperature.
8. Point 7 is repeated with an interval of 20 min in the first hour, 30 min -the second hour, 1 hthe remaining two hours, since at the beginning of shooting the greatest drift is observed. All time shooting 4 h. 9. Every 100 frames are averaged. The coordinates of the point centers are calculated. A drift chart of all 25 points is plotted.

Calibration of DMD with camera
1. Camera and DLP LightCrafter 4500 cool to room temperature.
2. Camera and DLP LightCrafter 4500 turns on. 100 frames of the DMD pattern are captured every minute within 4 h. After shooting, the DLP LightCrafter 4500 and camera turn off.
3. Every 100 frames are averaged. The coordinates of the point centers are calculated. A drift chart of all 25 points is plotted within 4 h with step of shooting 1 min, drift analysis in intervals 20 min in the first hour, 30 min -the second hour, 1 h -the remaining two hours.

Results Processing
The resulting images are processed according to a similar algorithm [6].
1. The signal value I n,m in each pixel n, m is averaged for captured 100 frames.
To reduce the effect of noise on average frames, a Wiener filter of 5 × 5 pixels is applied and a threshold restriction is introduced [6,9], signal values below which were taken equal to "0". The threshold value was 5 times the average value of the background signal.
2. The position of the image of one of the points of the DMD pattern on the photodetector is previously visually determined. Taking into account the known period between micromirrors [10] and magnification of system lens f ' = 500 mm-lens f ' = 1500 mm, the preliminary coordinates of the points of the DMD pattern on the photodetector are determined, which are used to set the center of the area for calculating the exact coordinates.
3. The exact image coordinates of each point of the DMD pattern are determined by the energy center of gravity: where I n,m -the signal value in pixel n, m; n, m -serial numbers of pixels in the columns and rows of the photodetector; N, M -the total number of columns and rows of the area of calculation of the coordinates of the center of gravity, an area of 70 × 70 pixels was used.  2020, vol. 11, no. 2, pp.
where p = 5.5 µm -pixel size of camera; v = 3magnification system of collimator lens-camera lens. 7. To analyze the relative drift, the value of the change in the distance along the rows and columns between neighboring points on the photodetector is calculated (characterizes how much the drift is the same throughout the array): where N s = 0, 1, 2, …, 4 row or column number. Similarly to p. 5, the value of the change in relative drift over time is calculated, Similarly to p. 6 to analyze the relative drift, the average value of the relative drift over all points, the RMS, and the maximum spread are calculated.
8. For a presentation of the drift in the plots, the drift value is increased by a selected coefficient k: 9. According to the obtained plots, values of average, RMS and maximum spread, one can judge the drift of DMD micromirrors and DMD micromirrors together with the pixels of a camera.

Analysis of the results
The drift of the points images formed by DMD micromirrors for 4 h in the form of a plot in the camera pixels is shown in Figure 4, its value is shown in Table 1. The analysis was carried out for 4 h, since this time is sufficient for the DMD and the camera to reach a stable operating mode, this time is sufficient for interior orientation parameters calibration of optoelectronic devices, and this time also covers the operating time of most optoelectronic devices. It can be seen that the points drift mainly along the lines. In addition to the point drift, one can note its slight non-uniformity for different parts of the DMD micromirror array.   2020, vol. 11, no. 2, pp. 122-131 M.A. Starasotnikau Приборы и методы измерений 2020 of optoelectronic devices, it is necessary to preheat the DMD until the drift of the micromirrors becomes such that it will not have a significant effect on the required accuracy of the interior orientation parameters calibration of optoelectronic devices. It is necessary to compensate for the value of the change in the period of micromirrors from the initial state of DMD. The analysis of Table 1 shows the predominance of column offset in the first 40 min of operation. The average drift after 20 min of operation on all points is 28 times greater than on the lines. The highest average column drift in the first 20 min of operation is up to 1.5 µm. The value of the drift of all points after 60 min of operation is less than 1 µm. The oscillation of the average drift after 60 min of operation is due mainly to random processes, primarily photodetector noise and vibrations. Thus DMD must warm up before calibrating optoelectronic device 60-120 min.
The displacement RMS of all points characterizes the uniformity of the DMD expansion, which will determine the accuracy of the interior orientation parameters calibration of optoelectronic devices using DMD, since uneven expansion will be disrupted the periodicity of micromirrors, that is, the relative position of the micromirrors with respect to each other will not be known. The value of the RMS of all points of DMD micromirrors is less than 1 μm. The maximum spread in the first 20 min of DMD operation in rows and columns is 2 µm, the rest of the time does not exceed 1 µm. Changing the distance between neighboring points also characterizes the uniformity of DMD expansion. From Table 2 it can be seen that the average change in the distances between neighboring points is no more than 0.1 µm. Thus, we can assume that the value of the change in the distance between neighboring points is the same over the entire array of micromirrors, is negligible and is random. The distance between neighboring points does not change over time. The residual effect of the uneven expansion of DMD during interior orientation parameters calibration of optoelectronic devices using DMD can be compensated by determining the coefficients of uneven drift for each DMD micromirror.

Table 2
The change value in the distance between neighboring points on the photodetector relative to the coordinates of the points on the photodetector of previous shooting intervals (in µm, recalculated to coordinate system of DMD) From Figure 5 the points drift within 3 h after 1 h of heating shows that the points drift 2 times more for the lower left part of the DMD than for the upper right part of the DMD. The points drift within 2 h after 2 h of heating becomes uniform. Residual nonuniform point drift can be caused by parallax, i. e. non-parallelism of the plane of the object formed by the DMD micromirrors and the plane of the camera photodetector.
The points drift due to the micromirrors of the DMD and camera pixels for 4 h in the form of a plots in the camera pixels is shown in Figure 6, its value is shown in Table 3. Table 3 shows the prevalence of column offsets in the first 120 min of work. The average drift after 20 min of operation on all points is 8.6 times greater than on the rows. The highest average column drift in the first 20 min of operation is 17 µm. When comparing with drift only due to the DMD from Table 1, an additional influence of the camera on the points drift appeared. The value of the drift of all points after 120 min of operation is 1 µm. Thus, the DMD and the camera must be warmed up during interior orientation parameters calibration of optoelectronic devices, heated for at least 120 min. When interior orientation parameters calibration another camera or Devices andMethods of Measurements 2020, vol. 11, no. 2, pp. 122-131 M.A. Starasotnikau Приборы и методы измерений 2020. -Т. 11, № 2. -С. 122-131 M.A. Starasotnikau any optoelectronic devices, it is necessary to study the drift of their photodetector matrix structure (or several photodetectors). However, the time for preheating will be approximately the same -120 min. The RMS value of the displacement of all points is less than 1 μm, except for the first 20 min, which is acceptable for most cases of interior orientation parameters calibration of optoelectronic devices. optoelectronic devices. In the first 20 min, the largest non-uniform expansion is observed; the RMS is 1.5 μm. The maximum spread in the first 20 min of operation on rows and columns is 4 µm, the rest of the time does not exceed 1 µm. From Table 4 the average change in the distances between neighboring points is not more than 0.1 μm. Thus, we can assume that the value of the change in the distance between neighboring points is the same, negligible, and random. The distance between neighboring points does not change over time.   Figure 7 the points drift within 3 h after 1 h of heating shows that the points drift mainly along the rows and more for the left side than for the top. The points drift within 2 h after 2 h of heating becomes uniform. The spread in the position of the same point is due to noise and vibration, a more pronounced spread for the lower right corner is due to residual parallax, i. e. DMD and photodetector planes are not parallel.

Conclusion
When using DMD as a test-object in problems of interior orientation parameters calibration of optoelectronic devices, the average drift value of the entire array of DMD micromirrors will affect the accuracy of calibration. In the course of the research it was found that the drift of DMD micromirrors occurs mainly along the rows in the first 60 min of operating. The value of the drift of all points after 60 min of operation is less than 1 µm.
The uniformity of expansion of the DMD will determine the accuracy of interior orientation parameters calibration of optoelectronic devices using DMD, since uneven expansion will disrupt the periodicity of micromirrors, that is, the relative position of the micromirrors relative to each other will not be known. It is determined that the RMS of the offset is less than 1 μm.
It has been established that the value of the change in the distance between neighboring points is the same throughout the micromirror array, is negligible, and is random in nature. If necessary, the residual effect of the uneven expansion of DMD during interior orientation parameters calibration of optoelectronic devices using DMD can be compensated by determining the coefficients of uneven drift for each DMD micromirror. Thus, DMD can be used as a test-object in the problems of interior orientation parameters calibration of optoelectronic devices. If more accurate interior orientation parameters calibration of optoelectronic devices are necessary, the results obtained can be used as compensation coefficients for changes in the spatial structure of DMD due to temperature effects during operation.