Application of 3D printing, sensor and artificial intelligence technology in Braille field

1. Introduction

With the development of technology and human being society, the problems of deaf-blindness people were gradually taken seriously. According to the statistic from National Center of Deaf-Blindness (NCDB) [1], one of 10,000 children in America diagnosed as deaf-blind. The article aims to design a translation equipment which can translate both speech to braille and braille to words. The equipment includes the application of 3D printing, sensors and artificial intelligence algorithms these three techniques. Among three techniques, 3D-printing can greatly improve the practicality and enforceability because 3D-printing technique use some specific materials to let the produce process more convenient and achievable [2]. The collecting of data and information basically responsible on sensors, the products will combine multiple sensors together and then use it to finish a complete tracking of motions or trails [3]. The artificial intelligence is responsible for all the calculation of information and the running of algorithm to detect the speech and finger patterns [4]. The reasonable application of three techniques will greatly help the deaf-blindness people have a better and more comfortable experience when they are communicating with healthy humankinds [5].

2. Related technology

There is a lot of ways to put 3D-printing into use with different technology using [6]. The 3D-printing technology this article going to mention is the vat photopolymerization [7]. vat photopolymerization is using a UV light to cures the resin layer by layer, it is very common seen when printing patterns and producing the prototypes. The biggest advantages using this technology is the high level of accuracy, braille can only be determined through touch feeling of a blind man so that production of the equipment must be very accurate in order to give the users of the equipment a high recognition rate of braille [8].

Sensors were used to do the sound detecting and collecting and motion tracking technology. Sound detecting and collecting need sound sensors with built-in microphone to receive a certain frequency range of sound wave so that it can transfer all the waves into electrical signal and let the AI system to cope with it. Motion tracking technology is another necessary technique for the product [9]. In order to complete a full motion tracking, there will be multiple sensors used. After the recording of deaf-blindness people’s finger pattern, the information will be sent to AI system and analyse what the deaf-blindness people trying to express [10].

3. Application analysis

   1. Education for blind

There are many uses of 3D-printing technology to give support on the education field of blind people. Due to the essay “Dedicated educational 3D printing sculpture teaching aids” [3], touching feeling is the main method to let blind people feel the identity of space and develop their thinking mode. The 3D-printing can produce sculpture teaching aids in both primary school and junior high that can help students read and study. For instance, there will be needed to produce the relief model such as the atom’s structure and stress concept diagram in physics and chemistry class in order to help blind students better understand the knowledge. The 3D-printing technology were also been proved that it has a practical use in the animation reversal teaching for deaf people. By introduced the concept of materialize abstract things, after scanning the 3D models the students made, the system will convert it into digital model and use 3D-printing to print the prototype of it.

According to the essay “[3]”, there are already actual use of 3D-printing technology from abroad. They use this technique emerged classical painting and sculpture made by the artist, the printing products can help blind and deaf students have a genuine sense of the grandeur and beauty of the work of art.

2. Braille writing pattern system

   1. The use of sensors on the products. Sound detection & collection sensor: Sound sensor is necessary in this product, during the progress of channel 1, the product need sound sensor to detect and collect the sound wave as a data, then send it to artificial intelligence which have specific algorithms to deal with it. There will be built-in microphone in the sound sensor, and it is used for collecting a certain amplitude longitudinal wave. Inside the mic, there will be a capacitor plate and another plate that are parallel to each other. When the sound wave hits the microphone, the capacitor plate starts to oscillate, and the capacitance will change. As a result, voltage across the two plates changes, in this case, we can control the capacitance in order to get some specific data.

Photoelectric sensors, angle sensors, IR sensors, optical sensors, accelerometers, inertial sensors, and magnetic bearing sensors. Theses sensors are all used for to do a motion tracking, the product have all the sensor built-in and use them to capture a complete motion. When the deaf-blindness people want to express something, they can use their finger to draw the pattern of a braille word in the air, and the camera placed on the products will record the video of the fingers motion. In order to successfully collect this data, all the sensors have to work together and achieve the motion tracking. In this case, the sensors can collect the data and send it to analyse and identify the patterns meaning through the built-in braille dictionary in AI part.

2. There are two channels of the systems. The first channel is translating from normal people’s speech to the touchable braille that deaf-blindness people can feel.

The second channel is translating deaf-blindness people’s finger moving pattern (braille pattern) into visible words that can display on the electronic screen. Channel 1: from normal speech to braille It helps to convey and translate normal language into corresponding Braille messages for the deaf-blind person to understand. Refer to the flow chart below. The microphone records the speech information as an analogy signal. As shown in Figure 1.

Figure 1. Sound processing flowchart.

The input signal is sent to an ADC (Analog to Digital Converter), which digitizes the data. In addition, the digital signal is then converted by MFCC (Mel Frequency Cepstrum Coefficient). The resulting data in the frequency domain shows unique features that can be compared with models trained in large databases. As a result, the corresponding text information is recognized. After receiving the text information, the matching Braille text can be identified by referring to the following mapping table. As shown in Figure 2.

Figure 2. Braille letters.

Channel 2: from braille pattern to normal words When deaf-blind people want to express their feedback to the outside world, they can draw the braille message on the trackpad. The input message will be interpreted through cycles letter by letter, which is illustrated by the below flow chart. (The explanation will use letter V as an example). As shown in Figure 3.

Figure 3. Braille recognition process.

The K-NN Algorithm is designed to realize the recognition of input pattern. In the database preparation phase, every Braille letter is simplified into a unique pattern. As shown in Figure 4.

Figure 4. L Braille.

For each unique pattern, 100 different samples that similar to the standard pattern is collected. As shown in Figure 5.

Figure 5. L-Braille sample.

Transform each sample pattern into 16x16 dimension filled with 1 and 0. White area is covered by 0, while black areas are covered by 1.Expand the 16x16 dimension into \(1 \times 256\) sequence of 0 and 1.In total, there are \(26 \times 100 = 2600\) samples in \(1 \times 256\) dimension that represented by 1 and 0. During the process of recognition, the system firstly receives the input pattern.Secondly, the input pattern is transformed into 16x16 dimension with 1 and 0. As shown in Figure 6.

Figure. 6 Finger pattern detected by sensors.

Then, expand the data to \(1 \times 256\) sequence of 1 and 0. Calculate the Squared Euclidean Distance between input data and every sample sequence in the Database:

\(d^{2}(n,s) = {(n_{1} - s_{1})}^{2} + {(n_{2} - s_{2})}^{2} + \ldots + \left( n_{255} - s_{255} \right)^{2} + {(n_{256} - s_{256})}^{2}\) (1)

If same number occurs at the same position, the distance increase by 0

\(0 \rightarrow {(1 - 1)}^{2} = {(0 - 0)}^{2} = 0\) (2)

If different number occurs at the same position, the distance increase by 1

\(1 \rightarrow {(1 - 0)}^{2} = {(1 - 0)}^{2} = 1\) (3)

Therefore, the smaller the distance, the higher the similarity of the input data to the compared sample. The default value of k is set to 50. According to the top 50 sample data that gives smallest value s of Squared Euclidean Distance, the Braille letter with highest occurrence is found. If 40 samples are related to the Braille letter V, the probability of the input data is letter V is 80%. Finally, the letter V will be sent to display screen and the system starts the next cycle.

4. Conclusion

Deaf-blindness is a condition that affects millions of people worldwide. People with this condition usually experience significant challenges when trying to communicate with others. This often leads to feelings of isolation, frustration, and a lack of connection to the wider community. Unfortunately, deaf-blindness is a condition that is often ignored by most people in society, and many people don't have the patience to engage in full communication with deaf-blind individuals. Without the ability to hear or see, deaf-blind individuals have an even more important reason to have fluent and direct communication with others. Communication barriers can limit their ability to participate in everyday activities, such as shopping, attending social events, or even traveling independently. This can lead to a sense of isolation and loneliness, which can impact their mental and emotional well-being. However, with the help of advanced technology, the society can become more inclusive and allow deaf-blind individuals to enjoy fluent conversations with others. For instance, the use of tactile communication methods such as Braille or finger spelling can greatly enhance communication with deaf-blind individuals, as these methods allow them to "see" and "feel" the message being conveyed. Furthermore, assistive technology such as text-to-speech software and speech-to-text software can help bridge the communication gap between deaf-blind individuals and others in their community. Artificial intelligence technology can also play a significant role in improving communication for the deaf-blind community. AI-powered devices can recognize and translate sign language into spoken language, enabling the deaf-blind person to communicate clearly with others. Additionally, AI-powered voice recognition tools can transcribe spoken words into text, making it easier for deaf-blind individuals to understand what is being said.

In conclusion, with the help of advanced technology, society can become more inclusive and accessible for deaf-blind individuals. The use of tactile communication methods, assistive technology, and artificial intelligence can greatly enhance communication and help to break down barriers. By making these tools more readily available, we can create a more inclusive society that allows deaf-blind individuals to fully participate in their communities and lead fulfilling lives. It is essential that we work towards creating a world where everyone has an equal opportunity to communicate and connect with others, regardless of any disabilities they may have.

References