ISAR Image Registration Based on Line Features

Article information

J. Electromagn. Eng. Sci. 2024;24(3):215-225

Publication date (electronic) : 2024 May 31

doi : https://doi.org/10.26866/jees.2024.3.r.222

Linhua Wu¹, Lizhi Zhao^1,, Junling Wang^2,, Jiaoyang Su¹, Weijun Cheng¹

¹School of Information Engineering, Minzu University of China, Beijing, China

²School of Information and Electronics, Beijing Institute of Technology, Beijing, China

^*Corresponding Author: Lizhi Zhao (e-mail: lizhi3285@126.com), Junling Wang (e-mail: emailwjl@gmail.com)

Received 2023 February 3; Revised 2023 June 29; Accepted 2023 November 13.

Abstract

I. Introduction

Inverse synthetic aperture radar (ISAR) plays an important role in space target observation and recognition by providing two-dimensional images of targets in all kinds of weather, as well as in day and night conditions [1]. ISAR image registration enables the analysis of target performance and behavior. By comparing the registered images from different views, researchers can monitor changes in the target’s structure, track its movements, and study its dynamics. This information is valuable for understanding the target’s capabilities, mission objectives, and operational patterns.

There are many methods for optical image registration and synthetic aperture radar image registration, such as normalized cross-correlation (NCC) [2], scale invariant feature transform (SIFT) [3], speeded up robust feature (SURF) [4], and a plethora of emerging deep learning methods [5]. However, it is difficult to migrate these methods directly into the ISAR field due to the different imaging mechanisms and imaging scenarios involved. The main differences can be summarized as follows:

· Limited information in ISAR images: Unlike optical images characterized by rich texture and gradient information and SAR images with richly detailed scenes, ISAR images usually involve several scatterers on a manmade target. In ISAR images, the most significant information is typically related to the brightness or reflectivity of the target, which is represented by the amplitude of the scatterers in an image.

· Low signal-to-noise ratio (SNR) in ISAR images: The complex electromagnetic scattering characteristics of the target, radar system errors, and translation compensation errors involved in practical imaging degrade image quality and reduce image SNR.

Research on ISAR image registration has mainly focused on the processing of three domains: signal, frequency, and image. Kang et al. [6] proposed a method for ISAR image registration in the signal domain, using raw echo data from different receive antennas to estimate the time-varying angular motion and compensate for the jointly translational motion to ultimately align the range and cross-range directions. Tang et al. [7] used the Fourier-Merlin transform to carry out phase conjugate multiplication in the frequency domain based on the Fourier theorem to achieve ISAR image registration. Furthermore, Gao et al. [8] utilized the correlation coefficient as a similarity measure to estimate alignment parameters. Meanwhile, Adel et al. [9] combined SIFT [10] and SURF [11], based on point features, to achieve image registration. However, in these methods, some meaningless noise points are often extracted as feature points, resulting in unstable performance. Since most space targets feature linear structures (e.g., solar panels and rectangular cavities), line features can be used to solve this problem, since they are more reliable and are less affected by the glinting or defocusing caused by scatterers.

There are many ways in which lines can be detected. The Hough transform (HT) [12] is a popular line detection technique that converts image space into parameter space, where lines are represented as points. However, it is computationally expensive and sensitive to parameter settings. Almazan et al. [13] proposed a probabilistic algorithm to detect lines and thereby improve the sensitivity of parameters, but it required large labeled datasets. Line segment detection (LSD) [14] is another commonly used method for line detection that is based on gradient information that does not rely on HT. Line segment merging (LSM) [15] is an upgraded version of the LSD that applies additional post-processing steps to merge overlapping line segments or remove redundant ones. Notably, line detection is often preceded by contour extraction, which helps segment foreground objects from the background, allowing for more accurate and focused line detection. However, the discontinuity and defocusing caused by scatterers in ISAR images may lead to the extraction of only a coarse contour, which can in turn degrade the performance of the line detection algorithm and lead to large errors in the registration results.

This study proposes a line feature-based ISAR image registration method that exhibits enhanced robustness against image degradation caused by noise, secondary reflections, and system errors. The proposed registration process involves line feature detection, feature matching, and parameter estimation. In particular, feature matching is accomplished by utilizing the spatial relations of line features between the reference image and the sensed image. Furthermore, the registration parameters are estimated by minimizing the sum of the Euclidean distances between the line features. Additionally, the accuracy of line detection in the LSM is improved by employing relevant techniques, such as image enhancement, contour extraction, and scatterer fitting. Moreover, the efficiency of parameter estimation is optimized by implementing the Snake Optimization (SO) [16] algorithm, which dynamically adjusts its search strategy and parameters, demonstrating solid capabilities in global exploration, robustness, and adaptability.

The remainder of this paper is organized as follows: Section II analyses the transform model for ISAR images from different perspectives, Section III introduces the proposed line feature detection and matching method, Section IV demonstrates the parameter estimation process of the ISAR image registration method, Section V provides the different experimental results to validate the efficacy of the proposed method, and Section VI presents the conclusions of this study.

II. Registration Model of ISAR Image Sequences

This section of the study uses a monostatic scenario as a reference point to analyze the relationship between the different viewpoints of ISAR images.

The ISAR imaging geometry is depicted in Fig. 1, where T_UVW represents the inertial coordinate system and T_XkYkZk is the imaging coordinate system at the k-th moment. Furthermore, X_k and Y_k represent the Doppler and range axes of imaging, respectively. Z_k is the normal vector of the imaging plane and e_los stands for the unit vector from the radar to the target. Notably, the relative motion of the targets with respect to the radar can be divided into translation and rotation. Translation must be compensated for before azimuth compression [17], while rotation is usually retained for ISAR imaging. Assuming ω_los represents the line of sight (LOS) rotation and ω_tar refers to the target self-rotation, the effective rotation vector can be formulated as:

Fig. 1

ISAR imaging geometry.

Furthermore, assuming translation compensation [18] is complete, the target motion can be considered equivalent to an ideal turntable model at a coherent processing time, as shown in Fig. 2.

Fig. 2

An ideal turntable model.

On considering the coordinates of scatterer Q on the target at the initial time as (x₀, y₀) and the range of the target center as R_Ok, the instantaneous distance of Q from the radar can be approximated as:

Here, c denotes the speed of light, B is the bandwidth, ω refers to the effective target speed, and the coherent processing time is t_obs. Furthermore, the range resolution is η_r = c/2B and the azimuth resolution is η_a = λ/2ωt_obs. According to the range-Doppler image algorithm, the position of Q relative to the center of the ISAR image can be obtained as follows:

Substituting Eq. (2) into Eqs. (3) and (4), the coordinates of Q in the image can be rewritten in the form of the following matrix:

Furthermore, on considering the image I₁ corresponding to t₁ as the sensed image, and the image I₂ corresponding to t₂ as the reference image, the position relationship of scatterer Q in these two images can be expressed as:

where γ = ω(t₂ − t₁).

However, offsets between images might still remain because of the errors introduced during the delay and the compensation process [19]. Therefore, supposing that the relative offset vector of the centers is p = [a, b], the transformation matrix can be expressed as:

The relationship between the same lines in the two images can be obtained in the same way. In this context, it is worth noting that, due to target scattering properties, noise, and low image quality, it is difficult to accurately determine the length of the lines. To address this, the direction angle and the vertical distance from the origin of the image to the line were utilized to describe a line, namely L = [ρ, θ], as shown in Fig. 2. Therefore, the position vector, which indicates the vertical distance vector from O_k to line L, can be formulated as:

Notably, the direction vector of line L is:

Therefore, the relationship between line L in the two images can be expressed as follows:

where r₁ and n₁ represent the position vector and direction vector in image I₁, respectively, while r₂ and n₂ denote the position and direction vectors in image I₂.

III. Line Feature Detection and Matching

ISAR images are collections of points, with the edge of the images being discontinuous. As a result, traditional automatic feature extraction methods present considerable challenges for ISAR images. In this section, the LSM algorithm is improved upon to extract line features from ISAR images, thus aiding in the subsequent image registration process.

2. Line Detection

LSM is well known for its efficiency and accuracy. The LSM process primarily comprises scale-space analysis, gradient calculation, line support region estimation, line validation, line pruning, and line mergence. Notably, the possibility of LSM will be further investigated in the experimental section of this study. In this paper, the outer contour image was used as the input for the LSM to extract coarse line features L = [ρ, θ].

In this context, it should be noted that during the course of this process, the outer contour errors were transmitted into the line errors. For example, the LSM result presented in Fig. 3 exhibits a gap between the LSM results and the natural edge of the target. To address this problem, the CLEAN technique [24] was employed to pick up dominant scatterers from the entire image and filter the scatterers located close to L = [ρ, θ]. Following this, the discrete scatterers were fitted to improve the accuracy of the LSM results. The following is a detailed description of the process adopted in this study for filtering scatterers and calculating line parameters.

Fig. 3

LSM results.

Assuming P_j(x_j,y_j) is the j-th dominant scatterer of the target and L_h = [ρ_h, θ_h] is the h-th line feature, the distance between P_j and L_h can be estimated using the following equation:

(11) dj,h=|tan (θh)xj-yj+ρh1+tan2 (θn)|1+tan2 (θh).

To calculate the distance between scatterer P_j and the other lines, and obtain an H-dimensional vector, d_j = [d_j,1, d_j,2, … , d_j,H], , where H denotes the number of lines. Subsequently, the association of P_j with a specific line can be determined based on an appropriate threshold ɛ_d = σ|η₁ − η₂|, where η₁ and η₂ are the dilation parameters involved in image preprocessing. Notably, σ can adjust the threshold value to affect the sensitivity of the association. If min (d_j) ≤ ɛ_d, P_j is considered to belong to the associated line, or it is excluded otherwise. Notably, the index of min(d_j) represents the line located closest to P_j.

After classification, the dominant scatterers in the same group can be linearly fitted. Assuming there are W points in the h-th group, with each point e represented as (x_w, y_w), the corresponding coefficient matrix after fitting will be:

(12) Fh=[∑xw2∑xw∑xwW]-1 [∑xwyw∑yw].

Subsequently, the fitted line L_h = [ρ̃_h, θ̃_h] can be obtained. Furthermore, the parameters ρ̃_h and θ̃_h can be formulated as:

(13) ρ˜h=|yj-Fh(1)xj-Fh(2)|1+[Fh(1)]2,

(14) θ˜h=arctan [Fh(1)].

3. Line Feature Matching

In image registration, establishing a corresponding relationship between line features is a primary task. For this purpose, the current study implemented a novel feature matching strategy to identify corresponding lines between two images based on geometric constraints. The initial stage of this feature matching process involved grouping based on angle information, while subsequent stages focused on matching the lines within the same group by ensuring consistency in the distance between them. The details of this process are as follows:

• Step I: Categorize the lines with similar orientations into groups based on their angle parameters θ. For instance, assume that I₁ (the sensed image) and I₂ (the reference image) are divided into M and N groups.

• Step II: Calculate the average angle of each group to attain θ̄_m, where m ∈ [1,2, … , M].

• Step III: Calculate the angle difference Δθ̄_m,n = θ̄_m − θ̄_m| between each group of I₁ and I₂. Store Δθ̄_m,n in a M×N matrix G_θ.

• Step IV: Analyze the statistical properties of G_θ and identify the correct match with the highest vote rate by applying a clustering algorithm. Then identify the approximate rotation parameter γ₀.

• Step V: Rotate I₁ using γ₀ as Ĩ₁.

• Step VI: Determine the intersection point of any two lines in the same image. Assuming q_i,j is the intersection point of L_i and L_j, the coordinates (X_qi,j, Y_i,j) of q_i,j can be calculated as follows:

(15) Xai,j=-ρj1+tan2 (θj)+ρi1+tan2 (θi)tan (θj)-tan (θi),

(16) Yai,j=ρjtan (θj)1+tan2 (θi)-ρjtan (θi)1+tan2 (θj)tan (θj)-tan (θi).

Remove any intersection points falling beyond the range of image I₁ and store the remaining intersection points in the set Q₁. Similarly, generate Q₂ for image I₂.

• Step VII: Choose one point from each point set (Q₁ and Q₂) to calculate the distance vector formed by the two points. Record the distance vectors to create a matrix G_q while iterating through all the intersection points.

• Step VIII: Perform cluster analysis on the distance vectors in G_q to identify the most frequent occurrences, ultimately completing the line matching.

After correctly matching the intersection points, the average distance vector of the intersection point pairs can be approximated as the translation vector p₀ = [a₀, b₀] between two images.

The key steps of the feature matching process are presented in the flowchart depicted in Fig. 4.

Fig. 4

Flowchart for feature matching.

IV. Parameter Estimation based on Line Features

This section presents the results achieved using the SO algorithm to estimate the image registration parameters. Taking γ₀ as the initial rotation angle and p₀ = [a₀, b₀] as the initial translation vector, the angle, range, and cross-range offset can be defined as Δγ, Δa;, and Δb. Therefore, the refined values can be considered as follows:

Furthermore, by substituting Eq. (17) into Eq. (10), the transformed line feature of L_h can be formulated as:

where θ̂_h,1 represent the angle of the h-th line in I₁ after transformation and ρ̂_h,1 denotes the distance from the origin to the h-th line. Consequently, the new position vector r̂_h,1 of h-th line in I₁ can be obtained.

The cost function is crucial for an optimization algorithm that guides optimization processes. In this study, the sum of the Euclidean distance between the position vectors of the lines was utilized as the cost function to measure the degree of image alignment. Therefore, assuming there are H_m pairwise line features, the cost function can be expressed as follows:

Notably, a minimum value of the cost function indicates that the two concerned images are well aligned. To improve efficiency, this study used SO to calculate the optimal solution for γ, a;, and b.

V. Experiments

To validate the superiority of the proposed method, different experiments were performed in three parts. The first part is concerned with proving the effectiveness of the proposed ISAR image registration method for simulated ISAR images, while the second part presents comparisons of the accuracy of different feature detection, optimization, and registration methods for ISAR applications using real measured images. In the last part, the robustness of the proposed method is analyzed through error analysis.

1. Effectiveness Validation

To validate the effectiveness of the proposed method, a number of experiments were carried out on simulated ISAR images, with the space target being RADARSAT-2, whose three-dimensional model is available in [25], as depicted in Fig. 5. The radar was set at Beijing (39.9 N, 116.4 E, 0 m), and the satellite two-line orbital element parameters were based on a set of public data available in the public satellite database [26]. The simulated images were generated using the range-Doppler algorithm. The main parameters of the ISAR imaging radar are listed in Table 1.

Fig. 5

Three-dimensional model of RADARSAT-2.

Table 1

Optimized dimensions of the 1 × 4 array

Two specific imaging times were selected from the observation views, as shown in Fig. 6. The ISAR images corresponding to the two views are presented in Fig. 7. In this context, it should be emphasized that the rectangular solar panels in both images are projected as lines, displaying limited information. Additionally, the reference image depicts an occurrence in which a solar panel obstructs the SAR antenna.

Fig. 6

The corresponding LOS parameters.

Fig. 7

Simulated ISAR images of RADARSAT-2: (a) reference image and (b) sensed image.

The line results detected by the proposed extraction method are depicted in Fig. 8, while the specific parameters of each line are listed in Table 2.

Fig. 8

Feature extraction results from the contours: (a) reference image and (b) sensed image.

Table 2

Details of the extracted lines

After feature matching, 12 association pairs were obtained—L_1,1L_2,1, L_1,2L_2,2, L_1,3L_2,9, L_1,4L_2,11, L_1,5L_2,12, L_1,6L_2,13, L_1,7L_2,10, L_1,8L_2,5, L_1,9L_2,6, L_1,10L_2,7, L_1,11L_2,8, and L_1,12L_2,3. Despite some obvious occlusions, the line features could be correctly matched. Most importantly, feature matching yielded the following rough register parameters: a₀ = 0.6438, b₀ = 0.3985, and γ₀ = −11.5629°.

In subsequent optimization algorithms, the search range of Δa and of Δb were set to [−100,100], while of Δγ was set to [−3,3]. Table 3 lists the estimated register parameters, while Fig. 9 depicts the registration results obtained by overlaying the two images—the one before and the one after registration. It is observed that although the main body and the solar panel are well aligned, the SAR antenna presents an alignment error caused by a projection difference between the two imaging planes.

Table 3

Registration results of the simulated images

Fig. 9

Comparison of overlapped simulated images before and after registration: (a) initial and (b) register.

2. Comparison Experiments

Comparison experiments were conducted using real measured ISAR images obtained from the laboratory's official website [27] (@ Fraunhofer FHR). The experiments were carried out in a Windows 10 Professional environment using Intel Core i7-1165G7 processors with 2.80 GHz speed and 16.0 GB RAM, while the codes were implemented in MATLAB R2021a.

2.1 Comparison of line feature extraction methods

The line detection results had a significant effect on the accuracy of the image registration. In particular, the results of three different line detection methods were compared based on the outer contour of the real measured ISAR images shown in Fig. 10(a). The line features extracted by conducting HT and LSM are presented in Fig. 10(c) and 10(b), respectively, in which the parameters of the distance merge while the shortest length limit of a line is the same. Notably, the HT results show more than two lines near certain areas. In both results, the lines do not fit the actual edges of the target, indicating failed ISAR image registration. In contrast, the result obtained using the proposed line detection method, presented in Fig. 10(d), shows that the line features align well with the target boundary, highlighting that it is more suitable and effective than both HT and LSM.

Fig. 10

Comparison of different line extraction methods: (a) outer contour, (b) HT, (c) LSM, and (d) the proposed method.

2.2 Comparison of optimization algorithms

An experiment was conducted to compare the efficiency of particle swarm optimization (PSO) [28] and SO under the same configuration. The iterate time was set to 200, and the parameters related to the learning rate in the SO and PSO were reduced to 0.6 times.

Fig. 11(a) shows the cost function value for each iteration. It can be observed that the PSO quickly converges to reach a local optimum. Additionally, the variation curves of the absolute register errors in terms of the number of iterations during the iterative process are shown in Fig. 11(b), 11(c), and 11(d). It is evident that the parameter estimation accuracy of the SO is better than that of the PSO. Furthermore, the time consumption of PSO and SO was 0.0488 seconds and 0.0324 seconds, respectively.

Fig. 11

Comparison of the optimization algorithm results: (a) cost function, (b) angle errors, (c) range errors, and (d) cross-range errors.

2.3 Comparison of different registration methods

To verify the superiority of the proposed method, real measured ISAR images featuring the same target were first obtained. Subsequently, the results of the proposed method were compared with those obtained using SIFT and artificial bee colony by carrying out normalized cross correlation (ABC-NCC) [29], with mutual information (MI) [30], normalized image correlation (NIC), and algorithm runtime as the performance indicators. The results are listed in Table 4, while the two overlapped images generated by the experiment are depicted in Fig. 12. It is evident that the proposed method is both fast and accurate. Fig. 12(d) shows that the main body in the image obtained using the proposed method almost overlaps with the real measured one, although the tail is displaced due to secondary reflections.

Table 4

Comparison of the proposed method with the different registration methods applied to real measured images of the spacecraft

Fig. 12

Comparison of different methods for registering real measured images of the spacecraft: (a) initial, (b) SIFT, (c) ABC-NCC, and (d) the proposed method.

Moreover, a different space object was also employed for image registration. The estimated register parameters are listed in Table 5, and the two overlapped images are presented in Fig. 13. It can be observed that the image produced by the proposed registration process successfully aligns with the target solar panel and accurately captures the change in the position of the SAR antenna.

Table 5

Comparison with the different registration methods applied to real measured images of ENVISAT

Fig. 13

Comparison of different methods for registering real measured images of ENVISAT: (a) initial, (b) SIFT, (c) ABC-NCC, and (d) the proposed method.

3. Robustness Analysis

To further analyze the robustness of the proposed method, 1,000 Monte Carlo simulations were conducted using different SNRs. Under the same scatterer model and mapping parameters, the SNR range was set as 10–30 dB, while the step size was considered 2 dB. Fig. 14(a), 14(c), and 14(e) present the mean error (ME) of the range offset, cross-range offset, and rotation of the images at different SNRs, respectively. Furthermore, Fig. 14(b), 14(d), and 14(f) present the root mean square error (RMSE) of the range offset, cross-range offset, and rotation, respectively. The results show that the proposed method is more robust than SIFT and ABC-NCC, especially with regard to the rotation angle estimation, which reached the order of 0.001. In fact, the main reason for the large errors observed in SIFT or ABC-NCC was that they were based on point or area features, while the ISAR images were noisy and sparse.

Fig. 14

Estimation errors of the simulation parameters at different SNRs: (a) ME of range offset, (b) RSME of range offset, (c) ME of cross-range offset, (d) RSME of cross-range offset, (e) ME of rotation angle, and (f) RSME of rotation angle.

VI. Conclusion

In this paper, a line features-based ISAR image registry method is proposed. First, a transformation model that incorporated both translational and rotational components was built, after which the traditional LSM was initially applied to detect the rough line features of the target contour. Subsequently, scatterers located in close proximity to the rough line features were fitted to refine and adjust the line features. The pairwise correspondence between the lines was examined using their spatial relations. Finally, considering the sum of the Euclidean distances between the lines as the cost function, the SO algorithm was employed to obtain precise parameters for ISAR image registration. The experimental results confirmed the effectiveness of the proposed algorithm in the ISAR image registration of different targets. Compared to existing methods, the proposed line features exhibited stronger robustness against noise and changes in image, as well as higher uniqueness and stability in matching, thereby improving the accuracy of the registration algorithms.

Article information Continued