Detection of Anomalies in Additively Manufactured Metal Parts Using CNN and LSTM Networks

Alireza Modir ^1,*, Arnaud Casterman ², Ibrahim Tansel ¹

1. College of Engineering and Computing, Florida International University, Miami, FL 33174, United States of America

2. Faculty of Engineering, The University of Mons, Mons, Belgium

* Correspondence: Alireza Modir

Academic Editor: Seyed Ghaffar

Special Issue: Additive Manufacturing Technology in Construction

Received: January 10, 2023 | Accepted: July 19, 2023 | Published: July 26, 2023

Recent Progress in Materials 2023, Volume 5, Issue 3, doi:10.21926/rpm.2303028

Recommended citation: Modir A, Casterman A, Tansel I. Detection of Anomalies in Additively Manufactured Metal Parts Using CNN and LSTM Networks. Recent Progress in Materials 2023; 5(3): 028; doi:10.21926/rpm.2303028.

© 2023 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.

1. Introduction

Metal parts are widely used in the manufacturing of critical components of machines and tools. Material waste and machining cost significantly increase the price of the metal parts. In addition, classical manufacturing methods require a long run time, sophisticated equipment, and qualified workers. The expansion of using end-product additively manufactured (AM) metal parts in many industries is expected in the upcoming years [1]. Beyond replacing conventionally manufactured parts, AM gives tremendous flexibility for the part geometry without increasing manufacturing cost and production time [2]. Many researchers use this technology to manufacture sensors [3] and embed transducers into structures [4]. Powder bed fusion (PBF) is one of the most commonly used AM methods. In this technology, an electron beam or a laser is implemented for fusing material powder together and a roller is used for spreading powder to the next layer. In this method, the part is fabricating layer by layer and moved down after each layer is completed.

Binder jetting and atomic diffusion additive manufacturing (ADAM) are new methods that have been getting popular since 2020. ADAM works with extrusion-based technology [5]. An ADAM-based 3D printer, Markforged Metal X System, was used in this study to fabricate AM test specimens. Metal powder and wax binder make up the filament used by this printer. Post-processing operations are needed to remove the polymer binder and reach the final mechanical properties [6]. AM structures are becoming increasingly common in different industries, and structural health monitoring (SHM) is needed to ensure the durability of these structures.

Methods for monitoring structural health can be classified into active and passive approaches. In the passive SHM, e.g. acoustic emission (AE) method, no external energy is applied to the structure, and damage is detected by monitoring the elastic waves generated by the damages in the structure. While in the active SHM approach, the structure is excited by a transducer and the electromechanical impedance or propagating waves are monitored for damage detection [7]. In the vibration-based damage detection SHM methods, the vibration response of the monitored structure is translated into meaningful parameters that indicate the structure's overall performance by identifying the presence, severity, and localizing of faults and damages [8]. These methods are classified into model-based and signal-based approaches [9]. In the model-based approach, the changes in the modal parameters such as the natural frequency and mode shape are used for damage identification [10]. In the signal-based approaches, statistical methods are employed to detect damages directly from the signal data. Monitoring the structural condition continuously requires employing data-driven approaches to handle large amounts of data. Machine learning algorithms have gained popularity among researchers in SHM studies. Analysis of the vibration response in time, frequency, and time-frequency has been widely used for feature extraction from the vibration response [11]. These methods require hand-crafted feature extraction by performing multiple signal processing and statistical analysis which is computationally costly [12].

Adams et al. [13] used a one-class Support Vector Machine (SVM) for damage detection in fiber-reinforced plastics using the electromechanical impedance approach. Elforjani et al. [14] used Artificial Neural Networks (ANN) and SVM for feature extraction from AE signals for damage detection in ball bearings. Through an ANN model, Hekmati et. al [15] used sensor data from both contact and contactless sensors to identify damages on a lab-scaled bridge. According to studies such as [16,17], hand-crafting features for inverse problems like SHM is not feasible. However, deep learning algorithms can automatically extract features from the data by learning a new representation of the input data in each layer with no need for human intervention [18]. Due to the limitations of classical machine learning, deep learning algorithms have gained attention in structural health monitoring studies [19,20]. The convolutional neural network is a deep learning algorithm that has demonstrated high accuracy in detecting objects and classifying images in several data benchmarks [21]. This model is currently being used by researchers in SHM to detect and localize structural damage [22,23]. Hung et al. [24] proposed a hybrid approach using a one-dimensional CNN and LSTM network for damage identification in a frame structure. In this method, the raw time-series data is fed into the proposed network for damage detection. The results showed that in the absence of noise, accurate damage detection can be obtained.

The 2D CNN classification can be achieved by converting 1D sensory data into 2D representations via signal processing techniques. In a Finite Element analysis, Gulgec et al [25] implemented CNN for damage detection and localization in gusset plates using strain distribution images. Wu and Jahanshahi [26] used CNN for the estimation and identification of linear and nonlinear structural responses in a three-story metal frame. The CNN has been used to detect damage in additively manufactured parts during and after production. Wen et al. [27] used CNN for the detection and classification of cracks and pores in laser beam powder bed fusion (LPBF) parts. For this purpose, in-situ images were collected during the AM process and a target detector model was used to achieve 90% damage detection accuracy. Cui et. al [28] used CNN for quality inspection of additively manufactured metal parts. Different architectures of the CNN network were investigated for different hyper-parameters for improving the detection of different abnormalities such as cracks, gas porosities, and lack of fusion. In a recent review study [29] authors overviewed different applications of CNN for quality monitoring of AM parts based on the type of input data (signals, images, or videos), the learning task, and the input data dimension.

AM structures are becoming increasingly common in different industries, and SHM studies are needed to ensure the durability of these structures. The majority of damage detection research in additive manufacturing (AM) involves in-situ monitoring during the manufacturing process. A review of ultrasonic testing applications in additive manufacturing was conducted by Honarvar et al. [30]. Nadimpalli et al. [31] proposed a sensitive ultrasonic non-destructive evaluation technique for in-situ monitoring of the bond quality of AM metal parts. Rieder et al. [32] used ultrasonic testing to monitor the layer-by-layer fabrication of a metallic test specimen using the Selective Laser Melting (SLM) technique. The X-ray Computed Tomography is used for comparison of the ultrasonic test with a CT-image. Strantza et al. [33] conducted a crack propagation study in additively manufactured titanium specimens using the acoustic emission method. In this research, a four-point bending test was employed for notched and unnotched specimens in a fatigue loading condition.

There have been few studies in the literature considering Lamb wave-based method for anomaly detection in AM components. A pulse-echo ultrasound-based study was conducted by Sol et al. [34] to investigate anisotropy in AlSi10Mg additively manufactured by selective laser melting. Using the second-harmonic generation in Lamb waves, Vien et al. [35] monitored fatigue damage accumulation in an AM aluminum plate at an early stage. Fakih et al. [36] used a scanning laser doppler vibrometer for monitoring the wave propagation and damage imaging in AM polymer parts with 20% and 100% infills. The investigation of infill density and skin thickness effect on lamb wave propagation can provide valuable insights for optimizing the ideal combinations of infill density and skin thickness that maximize the detectability of defects or abnormalities using Lamb waves. The results can help with accurate detection and characterization of defects in 3D printed parts. It can be used to assess the internal structure and quality of engineering structures.

As far as the authors are aware, there hasn't been any research on data-driven strategies for lamb-wave-based anomaly detection in AM metallic parts. Typical metal parts made by AM are smaller, less dense, and may have infills. The SuRE method is an active SHM method that has shown promising results in metal, composite, and polymer components [37]. In this technique the structure is excited with surface waves using a piezoelectric transducer and the dynamic response to excitation at the desired location is recorded using one/multiple piezoelectric sensors [38]. In this study, the SuRE method is used for recording the dynamic response to excitation in conventionally and additively manufactured metal bars. A comparison of wave propagation characteristics and the estimation of loading level are the objectives of this study. Based on the part’s internal geometry and print settings, we estimate the applied compressive load and classify it using CNN and LSTM models. In the following sections, the theoretical background, experimental method, results, and conclusion are presented.

2. Theoretical Background

The two variables that determine signals most fundamentally are frequency and time. The time domain refers to changes in amplitude over time, and the frequency domain refers to how frequently these changes occur. The Fast Fourier Transform (FFT) is one of the most widely signal processing tools which decomposes a signal as the sum of weighted sinusoidal functions of different frequencies, called frequency spectrum. One drawback of the FFT is that it does not give information about the time at which each frequency occurs. To characterize a signal's time-varying frequency content, joint Time-Frequency Transforms were developed. Short-Time Fourier Transform (STFT) and Continuous Wavelet Transform (CWT) are among the most popular time-frequency analysis methods. In essence, STFT is based on a moving window Fourier Transform over the time domain which generates signal representations that capture both the local time and the frequency content. The obtained image that represents time and frequency in two dimensions is referred to as a spectrogram. The discrete form of STFT for a discrete-time signal x[n] can be expressed as:

\[ X_{STFT}[m,\omega]=\sum\nolimits_{n=-\infty}^{\infty}x[n]g[m-n]e^{-j\omega n} \tag{1} \]

where g(m) is the window function at time m and $X_{STFT}[m,\omega]$ denotes the ω^th Fourier transform for the m^th time frame. Based on the window size, the resolution of the result can be good in either time or frequency. Here the frame index, m, is discrete but the frequency variable ω is continuous. By decreasing the width of the window, the resolution in time increases, while the frequency resolution decreases, and vice versa. Similar to STFT, the continuous wavelet transform (CWT) measures the similarity between an analyzing function (wavelets) and a signal using inner products. But unlike STFT which has a fixed window function, CWT implements a variable-size windowing technique. It uses long time intervals for capturing low-frequency information and short windows for better resolutions at high frequencies. Figure 1 shows different windowing approaches for the STFT and CWT. The wavelet transform of signal x(t) is defined by Equation 2:

\[ T(a,b)=\frac{1}{\sqrt{a}}\int_{-\infty}^{+\infty}x(t)\gamma^{*}\left(\frac{t-b}{a}\right)dt \tag{2} \]

where γ is the wavelet function and * denotes the complex conjugate, a and b are called the scale and translation parameters, respectively. In CWT the scales are discretized more finely compared to the discrete wavelet transform (DWT) which gives a more detailed time-frequency representation of the signal. In this study, the CWT function in MATLAB was used with a Morse wavelet with a symmetry parameter of 3 and a time-bandwidth product equal to 60.

Figure 1 An overview of the windowing approach for two transformations (CWT and STFT).

The 2D time-frequency representations obtained from using CWT and STFT transformations are stored in a database for training Convolutional Neural Networks for the classification of the structural condition. CNN is a Deep Learning algorithm that follows a similar adaptable nature as the human brain. It consists of several layers between the input and output which improves the identification of the dataset without requiring human intervention for statistical and probabilistic calculations. There are three main hidden layers in the architecture of a CNN: the convolutional layer, which is responsible for identifying the features of the image, the pooling layer, which is responsible for reducing the dimensionality in each step during the training, and the fully-connected layer which allows for continuous connections between features. Multiple blocks of convolution-pooling sequences can be used in the model depending on the complexity of the data, the computing cost, and the desired accuracy. In the convolution layer, matrix multiplication is used to convolve the input image with a filter with a much smaller size compared to the image. In this layer filters with learnable weights extract features from the input data. In the pooling layer, a fixed window is slid over the feature maps extracted from the convolution layer in order to select the most essential features that represent all the units captured by the window. Max-pooling layers are used to reduce computational cost and overfitting by downsampling the convolutional layer's filtered weights. Activation functions are added after the convolution layers for bringing non-linearity into the network. Using activation functions helps the network to learn the complexity of the data. Rectified Linear Unit (ReLU) is the most common activation function used in CNN. In contrast to ordinary neural networks, CNNs are not prone to overfitting since the convolutional layer and the pooling layer reduce the weights and the number of neurons. The two-dimensional convolutional layer and the one-dimensional Softmax layer are connected by a layer with the shape of a flattened vector, called the fully connected layer. The Softmax layer takes the characteristics from the fully connected layer, calculates the probabilities of each class, and outputs the class with the highest probability using a normalized exponential function.

In this study, a two-dimensional CNN was implemented for the classification of the recorded data. The workflow of the classification of the data is presented in Figure 2. The size of the input images is 583 × 514 × 3 in width, height, and color channels (red, green, and blue), respectively. The proposed architecture of the CNN used in this study is presented in Figure 3 which consists of three building blocks of convolution and pooling layers between the input data and the output. As the scalograms and spectrograms of the monitoring signal are very detailed, the dataset used in this study has high complexity, which makes using a CNN network with just one convolutional layer ineffective. Furthermore, the accuracy gain diminishes as the number of convolutional layers rises above four. The ReLU activation function was used in each block for introducing non-linearity to the network. The number of filters in the first, second, and third blocks are selected as 16, 32, and 64, respectively, with a kernel size of 3 × 3. In many cases, odd-sized filters are chosen, so the pixels from the previous layer surround the output pixel in a symmetrical manner. For the max-pooling layers in each block, the filters have the size of 2 × 2 with a stride of 2. Training's learning rate refers to the rate at which weights are updated. Although increasing the learning rate will allow the network to learn faster, it may not result in an optimal set of weights. In addition to increasing computation costs, lowering the learning rate to extreme values can result in permanent training errors. Based on the data from the Conventional part, CNN was trained with different learning rates and convolution blocks to determine the network architecture and hyperparameter values. Table 1 shows the averaged values of classification accuracy and training time based on four training sessions. In order to build the network, three convolutional blocks were selected and the learning rate was set at 0.001.

Figure 2 The workflow of the classification.

Figure 3 Architecture of CNN.

Table 1 CNN performance for different hyperparameters and network architecture.

LSTM is a Recurrent Neural Network (RNN) model which has gained attention for sequential tasks. LSTM algorithm has addressed the gradient vanishing problem in RNN networks by introducing a memory unit that helps with better handling of long sequential tasks and long-term information retention. There are obvious differences between the LSTM and RNN, with the LSTM consisting of interacting layers with a different mechanism of communication than a typical RNN gate. It consists of three gates (input gate, output gate, and forget gate) and an update status layer. LSTM networks are typically composed of memory blocks called cells. Each step transfers two states to the next cell, the hidden state ($h_t$) and the cell state ($C_t$). Data flow is controlled by the cell state, and sigmoid gates can add or remove data from the main chain of data flow (cell state). Similar to layers or matrices of operations, gates are composed of different weights in order to control the process of memorizing information by deciding which information is allowed on the cell state through learning which information is relevant, to keep or forget.

In the forget gate ($f_t$), the process of identifying and eliminating the unimportant data is performed by applying a sigmoid function ($\sigma$) on the output of the last LSTM unit ($h_{t-1}$) and the current input ($X_t$). The output of the forget gate is a vector containing 0 to 1 values representing the information pass rate (closing to 0 means to forget and closer to 1 means to keep). In the input gate, new information is either updated or ignored, in two steps: i) the values which need to be updated are obtained through a sigmoid function (values are transformed to the range of 0 to 1, where 0 means not important and 1 means important) and ii) using tanh function for squishing values between -1 and 1 to regulate the network. Figure 4 illustrates the structure of the LSTM network. The operator $\boldsymbol{\otimes}$ represents an elementwise product. The details of the LSTM network used in this study is presented in section 4.3.

Figure 4 The structure of an LSTM memory cell.

3. Experimental Setup

In order to compare the wave propagation characteristics of the 3D printed and the conventionally manufactured metal parts, one commercial stainless steel and two additively manufactured bars were prepared with the same dimension (180 × 38 × 6.5 mm³). It consists of one part printed along the largest surface which is called "the Horizontal part", and a second part fabricated along the second-largest surface which is called “the Vertical part”. The additively manufactured parts were fabricated using the Markforged Metal X 3D printer which uses the Fused Deposition Modelling (FDM) method also known as Fused Filament Fabrication (FFF) which incorporates Bound Powder Filament. All parts were manufactured using stainless steel 17-4PH filament. In order for a part to reach its final form, it must go through several stages, starting with CAD design. Figure 5 illustrates the workflow of the 3D printing of the metal parts. A printed part before postprocessing is referred to as a green part, and it is composed of metal powder, wax, and a binding substance. Parts are washed with a high-performance solvent, Opteon SF-79, to remove a high portion of the binder. In the washing process, most of the wax is removed from the brown part and the product is called the brown part. Finally, the heat treatment process is performed in the Sinter using a mixture of Hydrogen and Argon gases. As a result of this step, the part reaches its final dimensions, purity, and mechanical properties. Heat is applied to the parts to about 85% of their melting point, which transforms them from a loosely bound metal powder into a solid metal part [39].

Figure 5 The workflow of the Markforged metal printing process.

Figure 6 displays the three parts with fully solid infills used in this study. Four stainless steel bars with the same dimensions but different hidden geometries have been additively manufactured to study the influence of the infill pattern and skin thickness. Figure 7 shows the interior design of the parts with infills using the slicing software (Eiger). A 0.5 mm or 1 mm thickness of the skin was used with gyroid and triangular infill patterns. The layer height was selected as 125 μm for all the parts. Table 2 summarizes the dimensions, infill, skin thickness, and weight of the test bars.

Figure 6 Three different solid parts used in this study.

Figure 7 The internal view of the stainless steel bars with different infill patterns.

Table 2 Dimensions and characteristics of additively manufactured bars.

An illustration of the experimental setup used in this study can be found in Figure 8. Here, the SuRE method was used to excite the test specimen with ultrasonic surface waves and record the response to the excitation. In each specimen, two piezoelectric disks (SMD10T04R111WL) were attached to the opposing ends of each test specimen. An arbitrary function generator (Rigol DG1022) was used to provide a sweep sine wave in the range of 400 kHz to 450 kHz with a sweep time of 1 ms. To increase the signal-to-noise ratio, the maximum peak-to-peak voltage of the wave generator (20 V) was used for excitation. The part was excited in a wide range of frequencies and the highest amplitude was observed in the dynamic response to excitation in the range of 400-450 kHz. This excitation signal (a sweep sine in the range of 400-450 kHz with 1 ms duration) was selected to obtain the best compromise between the smallest sampling interval (to avoid aliasing without any filter) and the digital oscilloscope available memory without exceeding 100k data points. Different studies have confirmed that guided waves are strongly dispersive, which means that their characteristics such as phase velocity, energy, and amplitude change as a function of emitted frequency [40,41]. It was shown in [42] that it is possible to reduce the dispersion of the signal in the frequency domain by using a narrowband excitation signal. A digital oscilloscope (Owon XDS3104AE) with a 5 MS/s sampling rate was used to record the waves arriving at the piezoelectric sensor. The experiments were performed when each part was in a relaxed condition, with no external load on it, and when a compressive load was applied in the middle of the bars. Using a MARK-10 Series 5 force gauge, the load was applied at four different levels (50 N, 100 N, 150 N, and 200 N) and the experiment was repeated 10 times.

Figure 8 The experimental setup.

4. Results

The first objective of this research is to compare the ultrasonic wave propagation on the conventionally and additively manufactured stainless steel test bars. The second objective is to investigate the feasibility of the implementation of structural health monitoring methods for load-sensing in additively manufactured small parts. To be able to repeat the experiments without damaging the parts, the applied force is considered as a fault on the structure with different severity. The accuracy of the loading estimation was evaluated to determine the feasibility of the implementation of the SuRE method combined with deep learning algorithms. In order to compare samples based on their internal geometry, specimens were divided into two groups.

Solid bars formed the first test group; a conventionally manufactured bar and two additively manufactured bars 3D printed at different orientations. The bar was named “Horizontal” when its largest surface was built as the first layer on the build plate of the 3D printer. It was named "Vertical" when built from the second largest surface of the bar. The first test group included the bars numbered 1, 2, and 3 in Table 2 and the second test group included the solid Conventional bar and four additively manufactured bars with different internal geometries (numbered 1, 4, 5, 6, and 7 in Table 2).

4.1 Effect of Build Orientation and Infills on the Propagation of Acoustic Waves

A sweep sine wave was applied to the exciter PZT at one end, and the arriving waves to the other end of the specimen were recorded using another PZT working as a sensor. The results are plotted in Figure 9 when there was no load on the parts and when the maximum load (200 N) was applied at the center of each part. As the frequency of the applied signal increased linearly with time, the observed envelopes seem similar to the spectrums resulting from the Fast Fourier Transformation. In the first test group (top plots in Figure 9), the monitored signals for solid bars showed different characteristics, although their averages were almost the same. The results indicate that after applying force to the bars, the peaks appeared to decrease slightly. According to the bottom plots in Figure 9, the reached waves to the sensor in parts with internal geometries (infills) had much lower amplitudes than the solid parts. The infills change the material boundaries withing the part which can lead to wave dispersion. Another effect of infills on the wave propagation is the attenuation due to additional damping mechanisms in the part. Parts with thicker skin thickness tend to attenuate waves less than parts with thinner skin thickness. This also can be a response to material damping and energy dissipation.

Figure 9 The envelope of the time domain response for solid and infill parts in the relaxed condition and under 200 N compressive load at the center of each part.

For parts with a skin thickness of 0.5 mm (indicated by the red and black lines in the bottom row of Figure 9), the amplitude of response is considerably lower compared to specimens with a 1 mm thickness (represented by the blue and green lines). However, the variations in the amplitude of response resulting from changes in thickness are higher for parts with Triangular infills when compared to parts with Gyroid infills. Using the Horizontal solid bar as an example, Figure 10 illustrates the effect of load on the recorded response. In certain peaks and troughs of the monitored signal, the changes in amplitude are moderately higher, which allows deep learning networks to recognize these features to categorize the input data based on the applied load.

Figure 10 The envelope of the response for the Horizontal part five test loads.

4.2 Estimation of Applied Force and Type of the Test Bar Using 2D CNN

In five steps, the bars were subjected to forces from 0 to 200 N. The spectrograms and scalograms were calculated by using the STFT and CWT from the collected time-domain data, respectively. In this study, the sampling time was 0.04 µs and STFT was calculated by overlapping 512 FFT points by 20% with a Kaiser window. Scalograms were obtained using MATLAB's CWT function and a Morse wavelet with a symmetry parameter of 3 and a time-bandwidth product of 60. Data from the input images are cropped in a frequency range (350-500 kHz) to capture the most useful information for training CNN. Figure 11 presents the spectrograms and scalograms of the Horizontal solid bar. Color in these images represents the amplitude of vibration. Changes in color are a reflection of changes in amplitudes at some time-domain peaks and troughs. These changes can be detected by filters in each layer of the CNN as features of data classification. Some of these differences are highlighted in Figure 11. For spectrograms and scalograms, separate 2D CNNs were trained using 70% of the images for training and 30% for test.

Figure 11 CWT and STFT images of the Horizontal part under 200 N compression and relaxed conditions.

In the first study, the additively manufactured and commercially manufactured bars were subjected to 5 levels of load and the time-domain data were collected 10 times at each loading condition. CNN model was used to estimate the applied force and type of the test bars based on the time-frequency representatives of the sensory data. CNN was trained on 105 samples and tested on 45 samples for each image type (spectrograms or scalograms). The training process was repeated 4 times with a random split of data for training and testing. The results are averaged and tabulated in Table 3. It can be seen that CNN had a slightly better performance using spectrograms. Figure 12 shows a sample of the classification results for each of the image types. As a point of interest, all of the mislabeled cases were within 50 N of the actual load level in that case. For simplicity, the Conventional, Horizontal, and Vertical parts were referred to as C, H, and V, respectively. Five different categories for each bar represented by 0, 1, 2, 3, and 4 correspond to 0 N, 50 N, 100 N, 150 N, and 200 N loads, respectively.

Table 3 The performance of CNN for both loading level and orientation classifications.

Figure 12 CNN classification results for solid parts under 5 different loads.

For the second study, CNN training was repeated four times for each input dataset, spectrogram and scalogram of the parts with triangular and gyroid infills. The architecture of the network and the hyperparameters are kept constant for this study. The dataset for each part with infill consists of 10 classes, two different skin thicknesses, each at 5 different loading conditions. The purpose it to estimate the skin thickness of the part as well as the applied load. Table 3 presents the average classification accuracy for each of the case studies using 70 images for training and 30 for testing. Figure 13 illustrates the test results of one training process for the cases which were not used in the training process. The name of each category is presented with four characters, the first two characters (T1 and T2) refer to the parts with triangular infills with 0.5 mm and 1 mm skin thickness respectively. The following two characters (L0, L1, L2, L3, and L4) correspond to the 0 N, 50 N, 100 N, 150 N, and 200 N loads, respectively. This study was repeated for the bars with gyroid infill and the results are presented in Figure 14. The first two characters (G1 and G2) in the name of the class refer to the gyroid infills with 0.5 mm and 1 mm skin thickness, respectively. Here also the misclassified cases were within 50 N of the actual load level.

Figure 13 Classification results for bars with triangular infill under different loadings.

Figure 14 Classification results for bars with gyroid infill under different loadings.

4.3 Estimation of Applied Force and Type of the Test Bar Using LSTM

LSTM has been suggested in the literature for the prediction and classification of time-series data. The raw vibration response data in the time domain were fed into the LSTM network for training and testing. Hence the input size for the sequence input layer would be one. The next layer in the network is a bidirectional LSTM (Bi-LSTM) layer consisting of 150 hidden units. Bi-LSTM is a variant of LSTM which presents each training sequence twice, forward and backward, to two independent recurrent networks which help to retain information from the past and the future. A fully connected layer, a softmax layer, and a classification output layer are the three final layers of the network which connect the hidden units in the LSTM layer to the output label classification. An adaptive moment estimation (Adam) optimizer was used for updating the network weights during the training process. The training process continued for 200 epochs with an initial learning rate of 0.001. Similar to CNN, 70% of the data was used for training the network and 30% for the test. In order to assess only LSTM's pure generalization capability, its architecture is fixed for each training process.

For each of the three solid parts, the proposed LSTM architecture was trained to evaluate the estimation of the load level, resulting in 5 classes of data. This process was repeated for parts with similar infill patterns (2 parts with triangular infill and 2 parts with gyroid infills) resulting in 10 classes of data for each training. The results of the test data for each of the cases are tabulated in Table 4. An output classifier's quality is typically measured with a confusion matrix. The confusion matrix plotted in Figure 15 provides a visual representation of the network's performance in estimating the load level on the Conventional part, as an example. Out of 15 test cases, only 1 case was misclassified which led to an overall accuracy of 93.3%. It can be seen that CNN results outperform LSTM when the number of classes increases.

Figure 15 LSTM classification results for the Conventional part under 5 different loading.

Table 4 Performance of LSTM on loading estimation accuracy for all parts.

5. Discussion

The use of deep learning techniques in vibration-based structural health monitoring of 3D printed parts is a novelty study for several reasons:

• Limited research: The field of 3D printing is still relatively new, and research on structural health monitoring of 3D printed parts is limited. While there have been some studies on the use of vibration analysis to detect defects in 3D printed parts, the application of deep learning techniques in this context is still rare.
• Complex data: Vibration data collected from 3D printed parts can be complex and difficult to interpret. Deep learning techniques, such as convolutional neural networks and recurrent neural networks, have shown promise in processing and analyzing complex data.
• Enhanced accuracy: Traditional methods of structural health monitoring may not be sufficient to detect defects in 3D printed parts. Deep learning techniques can enhance the accuracy of defect detection by identifying subtle changes in vibration patterns that may not be apparent to the naked eye.

Overall, the application of deep learning techniques in vibration-based structural health monitoring of 3D printed parts is a novel and promising area of research that has the potential to improve the quality and reliability of 3D printed parts. Here are some suggestions for future research:

• Due to the limitations of the additive manufacturing facilities, most of the test specimens were smaller than 50 cm in length. It is suggested to evaluate the application of the proposed SHM method for damage detection and load sensing in parts with higher dimensions and more complex geometry.
• In this study, CWT and STFT were used to represent the monitored signals' time-frequency characteristics. A comparison between the other signal-to-image transformation methods should be taken into consideration when using 2D CNN for the classification of the data. 
• The data augmentation methods used in this study were applied to the time domain data before converting them to the time-frequency representations. Since the horizontal and vertical axis in spectrograms and scalograms carry important information, using conventional image data augmentation such as mirroring and rotating may not be applicable to these datasets, generative adversarial networks (GAN) may be used to generate artificial data for expanding the size of the dataset. 
• Although every little detail in the test specimen such as the piezoelectric location, and the glue thickness may change the monitored dynamic response to excitation in identical test specimens. Limited test cases in experimental studies can be overcome by adding finite element simulation data in order to increase the dataset size for better classification results.

6. Conclusions

In this paper, the SuRE method was implemented to monitor the ultrasonic surface wave propagation on metallic rectangular bars manufactured conventionally and additively with different print orientations and infill patterns. Also, two deep learning algorithms were implemented for evaluating the load-sensing capability of this SHM method. One conventionally made and six additively manufactured stainless steel metal bars with and without infills were fabricated for this study, all with the same dimensions of 180 mm × 38.5 mm × 6.6 mm in length, width, and height, respectively. A sweep sine wave in the range of 400-450 kHz was used for the excitation of the bars with surface waves from one end of the specimen and the dynamic response to excitation was monitored at the other end. Compressive loads in the range of 50 N to 200 N with 50 N steps were applied at the center of each specimen. Applied load at the center of parts changed the amplitude of monitored signals at certain peaks and troughs. The envelopes of the monitored signals for solid specimens showed that although their characteristics were different, their average amplitudes were similar. Signal amplitude for parts with infills was significantly lower than that for solid bars, showing that infills reduce the energy of propagating waves. As a result of this energy loss, parts with thinner skins encountered higher losses.

In order to classify the recorded sensory data for the load level and manufacturing characteristics of the parts, two-dimensional convolutional neural networks (2D CNN) and Long short-term memory networks (LSTM) were implemented. The recorded data in the time domain were converted to images in the time-frequency domain using STFT and CWT transformations. The dataset was divided into two main groups for parts with totally solid infills and parts with infills. Spectrogram and scalogram obtained from CWT and STFT transforms were used as input data for the CNN while LSTM used the raw time-series data for estimation of the loading level and manufacturing characteristics. The results obtained from CNN showed that using spectrograms for training CNN resulted in slightly better overall classification accuracy. The results revealed that in studies with more classes, LSTM's performance decreased compared to CNN. The results for LSTM show that 7 and 6 cases were misclassified for parts with Gyroid and Triangular infills, respectively, while with CNN there were between 2 and 4, depending on the input type.

According to the results, the SuRE method can be used to estimate the applied loads on small metal structures. Bars made by conventional methods and additive manufacturing perform similarly. In parts with totally solid infills, surface waves travel with greater amplitude than in parts with infills. Wave amplitudes decrease even more with decreasing skin thickness for parts that have infills. CNN was found to be an excellent tool for categorizing data based on manufacturing methods and load levels. A good classification performance was seen in all test cases using CNN, with spectrograms outperforming scalograms.

Author Contributions

Conceptualization: Dr. I. Tansel, Dr. A. Modir, Methodology: Dr. Modir, Dr. Tansel, Data Collection: A. Casterman, Dr. A. Modir, Writing - Original Draft: A. Casterman, Dr. A. Modir, Writing - Review & Editing: Dr. A. Modir, Supervision: Dr. I Tansel, Dr. A Modir.

Competing Interests

The authors have declared that no competing interests exist.