You are on page 1of 8

Volume 8, Issue 2, February – 2023 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Electronic Speaking System for Speech-Impaired


People Using Raspberry Pi Pico
Rukhsar Shaik*, Nadakuditi Harshitha*, Gayathri Vasamsetti*, Velagapudi Bala Krishna*, Goru Hema Kumar**
** Department of Electronics and Communication, V.R. Siddhartha Engineering College,
Vijayawada 520007, India

Abstract:- Around 2.78% of people in our nation have


trouble speaking clearly. It is challenging for these folks
to interact with others. When engaging with others, they
feel as though there is a communication gap, which
prevents them from openly expressing their emotions
and sentiments. The suggested solution is to create a
hand-assistive system for physically handicapped people.
The goal of this initiative is to address the different
issues that physically disabled persons confront. The
major goal of this research is to lessen the
communication gap between people who are physically
disabled and people who aren't. The components of this
gadget include gloves, sensors, a Raspberry Pi Pico, a
speech module, and a 16x16 LCD screen. Essentially, the
voice module will translate the movements into speech in
real-time. Gloves, sensors, a Raspberry Pi Pico, a speech
module, and a 16x16 LCD make up this gadget. The Fig 1 Sign Language or Plains sign Talk
movements will essentially be translated into real-time
speech produced by the voice module, and the display Fig 1 depicts the different hand gestures of the sign
will output text for each corresponding gesture. language. Few people in the remainder of India's population
Therefore, this device offers both deaf and dumb and can use Indian Sign Language because they don't need to
sighted persons an effective means of communication. learn it. This makes it difficult for the Deaf Dum Band.
Normal people communicate with each other. As a result,
Keywords:- Raspberry Pi Pico, HC-05 Bluetooth Module, hearing-impaired people are excluded from society's
Voice Modulator, Python. mainstream. A communication assistant is needed to
translate sign language into aural speech to issue. Some
I. INTRODUCTION systems have been created in the past to accomplish the
same goal, but they had drawbacks such as not being
The biggest setback in a person's life is a disability. A total portable, almost impossible to implant, or expensive. The
of 26 million people in India are disabled, according to data only goal of all previous systems was one-way
from the 2011 census. People who are physically unable to communication.
communicate can only do so through sign conventions or
with the assistance of interpreters. The challenge with sign Thus, to fulfill this demand in this study, we employ
language is that a normal person might not understand it at the Raspberry Pi Pico model. First, the code is in Arduino,
all, making it difficult or impossible for physically disabled and after certain simple gestures are made by stupid people,
persons to communicate with one another. Making them Python technology will be used to play the audio. The
independent and allowing them the freedom to speak with system, which is based on Indian Sign Language, has all the
regular people without the need for interpreters is thus made vocabulary needed for daily life. Additionally, there is a
feasible by creating a system that enables them to do so special, tailored section for personal data. A language
without interruption, allowing them to express their processing algorithm is created to organize the words into
sentiments and emotions. This project's primary goal is to grammatically correct English sentences due to the
create an embedded system that includes a Raspberry Pi differences in sign language grammar.
Pico, hand gloves, sensors, an accelerometer sensor, a speech
module, and a speaker.  Disability in India-Fast Facts:

 According to the 2011 Census, 121 million people are


living in India; 2.68 billion of them are classified as
"disabled" (2.21% of the overall population). Out of
2.68 crores,

IJISRT23FEB939 www.ijisrt.com 1455


Volume 8, Issue 2, February – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
 1.5 crores are males and 1.18 crore are females  Field Survey:
 The majority (69%) of people with disabilities lived in
rural areas.
 Sikkim has the highest prevalence of disability among
the states and UTs(2.98).
 The lowest prevalence of impairment (0.9%) was seen
in Daman and Diu.

 Types of Disability;

Fig 3 Field Survey with the Students

Fig 2 Disabled Population by Type of Disability in India-


Census,2011

Fig 2 represents the various types of disability


problems such as vision, speech, movement, and mental
retardation. According to the census 2011, disability in
movement represents the highest among all problems.
Fig 4 Interaction with the Students
II. RELATED WORK
Our team went to Madonna Deaf and Dumb School in
Ryan Patterson invented the first-hand Talk glove in Ramavarapadu, Krishna District, Vijayawada, Andhra
2001 [1]. With his signing, he started his assignment. The Pradesh, to survey the speech-impaired students. The school
two independent devices that make up a signing interpreter has four classes with a total enrollment of 200 students.
are a glove with flexible sensors sewn into it that track the Students range in age from 5 to 15 years. The students of
position of the fingers by measuring the electric resistance X- standard interacted with the team. Students
created as the fingers bend [1]. The shift in electrical current communicated with us through the blackboard and hand
is converted into digital signals by a tiny microcontroller on gestures. The students interacted with us actively and are
the back of the hand, which then wirelessly transmits them eager to share their experiences. During the conversation, it
to a computer. The computer then interprets the arithmetical is discovered that some students have issues with their vocal
values into the letters that display on the screen. The biggest cords. As a result, the idea of hand gestures using hand
drawback of this type was that it could never function gloves and a Raspberry Pi Pico is proposed, which is both
without a computer or laptop, which reduced its portability inexpensive and portable. Figure 3 and 4 depicts the
[1]. Hand Talk Assisting System for the Deaf and Dumb, interaction with students.
embedded-based In March 2014, a method was created. This
solution makes use of a simple keypad to store and play The system's operation and data processing are done
audio. The disadvantage of this method is that signing is not on an Arduino board. The device helps silent persons
used. This process was created in May 2014. With the use of transmit fundamental signals by using roughly 16 recorded
MATLAB, this method turns the gesture into audio. The messages like "Help," "Danger," "Hungry," and so on. The
system's biggest flaw, however, is that it is not portable and system interprets user finger gestures for various hand
constantly needs a computer for conversion. Talk Aloud movement variations. Sensor input values are continuously
Gloves. This technique was developed in April 2016[2]. received by the Arduino, which then processes them. Now it
This system senses gestures with the flex sensors and audio looks for messages that match a specified pattern for the set
is produced with the assistance of a computer. The main of sensor data. When the message is located in memory, it is
drawback of this technique is that it requires a computer retrieved and spoken out using a smartphone app. So,
and there's no way for another person to speak. utilizing a straightforward wearable device, we now have a
fully functional smart-speaking system to assist mute people
in communicating with non-mute people.

IJISRT23FEB939 www.ijisrt.com 1456


Volume 8, Issue 2, February – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
III. PROPOSED MODEL  Voltage Divider Formula.

The Raspberry Pi receives the power supply as an


input. The system receives the hand motion as input. The IR
sensor picks up and registers the hand gesture. The
Raspberry Pi can store several commands. The Raspberry
Pi's memory is compared to the input command that was
delivered. Given the input, the output of the HC-05 IV. HARDWARE SPECIFICATIONS
Bluetooth module will be adequate. The PCB works as a
voice module. The Raspberry Pi's command is transferred to A. Raspberry Pi Pico
the PCB, which functions as a voice module, where it is A new microcontroller from the Raspberry Pi
executed. A mobile Bluetooth text-to-speech converter foundation is called the Raspberry Pi Pico, or just Pico. It
program transforms this voice into text. offers a dual-core ARM processor, 2 MB of flash memory,
and 26 GPIO pins on the hardware side. Software-wise, it
provides a port of Micro Python as well as a comprehensive
C/C++ SDK. Figure 6 depicts the Raspberry Pi Pico's pin
diagram.

Fig 5 Proposed Block Diagram

Specifically, the Raspberry Pi Pico is provided with a


power source, a speech modulator, and an 8-ohm speaker.
These components are connected via the Raspberry Pi Pico's Fig 6 Raspberry Pi Pico
GPIO pins, an ABCD pin-based IR sensor, and a Bluetooth
HC-05 module. The proposed model for hand gestures using  The Pico has the following features:
the Raspberry Pi Pico RP2040 microcontroller and The Raspberry Pi microcontroller chip was created in
APR33A3 voice recorder and playback IC with an 8-ohm theUK as the RP2040.
speaker that is coupled with GPIO [8,9,10,11,12,13] Pins is
illustrated in Figure 3. It's known as a voice module. The  ARM Cortex-M0+ dual-core processor with a variable
Raspberry Pi Pico is also connected to three IR sensors, clock speed of up to 133 MHz
which are connected to ABCD [26, 27], and there are three  SRAM memory of 264 kB and on-board flash memory
ABCD pins on the Raspberry Pi Pico. In place of the LCD, of 2 MB
we are using a Bluetooth HC-05 module, which is connected  Soldering modules directly to carrier boards is possible
to a mobile device via one Bluetooth DTS; the GPIO with casted modules.
[4,5,6,7] pins we are using for this project, which are GPIO  Support for USB 1.1 Host and Device
[14, 15, 16, 17] pins, were also used. The Raspberry Pi is  Dormant and low-power sleep modes
connected to the power source via a bridge rectifier, a filter  Programming with drag-and-drop and large storage
capacitor, and 7805 transistors. Using a voltage divider rule, through USB
signal conditioning is accomplished. Flex sensors are  26 GPIO multipurpose pins
coupled to IOK resistors. Flex sensors' resistance alters  SPI, I2C, UART, three 12-bit ADCs, and
with each bend, and for each gesture, a certain voltage 16programmable PWM channels are all included.
output result.
 On-chip accurate clock and timing [3].

IJISRT23FEB939 www.ijisrt.com 1457


Volume 8, Issue 2, February – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
B. HC-05 Bluetooth Module  RX Pin
The HC-05 module as shown in Fig 7 is a Bluetooth In UART communication, this pin will be the data-
Serial Port Protocol (SPP) module that provides a wireless receiving.
and transparent serial data link between two devices. It
operates in the 2.4 GHz ISM frequency band and supports  State Pin
Bluetooth Version 2.0 with Enhanced Data Rate (EDR) The state will display Bluetooth's current state. It
for faster data transfer rates up to 3 Mbps. informs the controller about Bluetooth connectivity with
another device.

 Enable/Key Pin
The only pin that distinguishes the HC-05 from the
others is the enable/key pin. It is useful to use an external
signal to switch the device between data and command
mode.

 Button Pin
The command and data mode states can be changed
using a button on the module.

 LED Pin
The LED will aid in visualizing the various states of
the HC-05 module [4].
Fig 7 HC 05 Bluetooth Module
C. IR Sensors:
This module includes a complete radio transceiver A sensor that produces infrared light to sense certain
and baseband, making Bluetooth connectivity simple to features of its environment is an electrical gadget. Both the
implement in a wide range of embedded systems and heat and motion of an item can be measured by an IR sensor.
devices. The module is powered by a 3.3- to 5-volt power These kinds of sensors, also known as passive IR sensors,
supply and works with a variety of microcontrollers and don't emit infrared radiation; instead, they merelymeasure it.
other embedded systems. A popular Bluetooth module Typically, all objects in the infrared range emit some kind of
known as the HC-05 can provide two-way (full-duplex) thermal radiation.
wireless connectivity to your projects.

 HC-05 Technical Specifications:


The following specifications describe the Serial
Bluetooth module for Arduino and other microcontrollers:

 Operating Voltage: 4V to 6V (typically +5V)


 Operating Current: 30 mA
 Range: approximately 100 m
 Compatible with Serial communication (USART)and
TTL
 Follows IEEE 802.15.1 standard protocol
 Uses frequency-hopping spread spectrum (FHSS)
 Can operate in master, slave, or master/slavemode

 HC-05 Pinout Configuration:

 VCC Pin Fig 8 IR Sensor Pin Diagram


HC05 Modules, like any other device, require power
to function, and the VCC pin interfaces with the external Fig 8 is an IR Sensor pin diagram in general, an IR
power source. sensor will have two important pins signal and power.
Infrared sensors can be further classified into IR transmitters
 GND Pin and receivers. IR transmitters emit infrared radiation, while
Every device requires a common ground to interface IR receivers detect it. IR transmitters are commonly used in
with the microcontroller. remote controls and other applications where a signal needs
to be sent wirelessly. IR receivers, on the other hand, are
 TX Pin used to receive signals from IR transmitters and are often
The HC-05 Bluetooth module communicates with the found in devices like TVs and set-top boxes.
microcontroller via UART. In UART, the TX will be the
module’s data transfer pin.

IJISRT23FEB939 www.ijisrt.com 1458


Volume 8, Issue 2, February – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
 Active IR Sensor  Bridge Rectifier
This sort of sensor consists of a transmitter and a This component converts the transformer's lower-
receiver, sometimes known as an emitter and a receiver. A voltage AC output into pulsing DC voltage. The bridge
laser diode or LED is typically employed as a source. While rectifier is made up of four diodes connected in series to
laser diodes are used for imaging infrared sensors, LEDs are ensure that the output voltage is always positive.
used for non-imaging infrared sensors.
 Capacitor
An infrared sensor can function by radiating radiation By storing and releasing electrical charge, this
that is then detected and received by the detector. To component smoothes out the pulsing DC voltage from the
obtain the necessary data, it is further processed using a bridge rectifier. The capacitor is connected in parallel to the
signal processor. Reflectance and break-beam sensors are the bridge rectifier's output.
top examples of active infrared sensors.
 Regulators
 Passive Infrared Sensor These components maintain the output voltage at a
Only detectors are used in passive infrared sensors constant level, ensuring that it remains stable even when the
(PIR), which use targets like infrared transmitters or input voltage and load conditions change. The 7805 voltage
sources. Here, the object will emit energy, and infrared regulators, which provide a fixed 5-volt output, are common
receivers will pick it up. The signal is then decoded by a voltage regulators used in power supply circuits.
signal processor to produce the needed data [5].
 Output Voltage
D. Power Supply This is the power supply circuit's final output, which
An electrical circuit known as a power supply is provides the necessary DC voltage levels for electronic
created to deliver different ac and dc voltages for device devices.
functioning. A variety of source voltages isnecessary for the
proper operation of electronic equipment. Transistors and  Power Supply Functions:
ICs must work at low dc voltages. To run CRTs and other The entire power supply circuit can carry out the
devices, high voltages are required. All of these voltages can following tasks:
be generated by batteries. However, the neighborhood power
company typically provides electricity for electrical and  By using a transformer, step voltages are raised or
electronic equipment. This electricity is delivered by an lowered to the desired ac line voltage.
outlet at 115 volts AC and 60 hertz. Some equipment  To fulfill the needs of the equipment, offer a voltage
requires a different voltage to operate. A bridge rectifier, division technique. Either half-wave or full-wave
filter capacitor, 7805 regulators, and lead are used in this. rectification can be used to convert ac voltage to pulsing
dc voltage.
 Filter pulsing dc voltage to obtain a pure, constant dc
voltage for equipment use.
 Adjust the output of the power supply by theapplied load
[6].

E. Voice Module
The Voice Recognition Module is a small, easy-to-use
speaking recognition board. This speaker-dependent module
supports up to 80 voice commands. Any sound can be
trained to function as a command. Before the module can
recognize any voice command, users must first train it.
Voice commands are grouped in a large group, much like
books in a library.

Fig 9 Power supply circuit diagram In this case, the APR33A3 speech module is used. The
APR33A series IC, a powerful audio processor, is coupled
 Fig 9 Shows the Power Supply Circuit Diagram and with high-performance analog-to-digital converters (ADCs)
the Components as Follows: and digital-to-analog converters (DACs) on the 8-Channel
Voice Record and Audio Playback Board8.5
 Transformer
This component converts high-voltage alternating
current (115 VACS, 60 Hz) to a lower-voltage alternating
current level (typically 12-24 VAC). The transformer is
made up of a primary coil that is connected to the input
voltage and a secondary coil that delivers the lower voltage
alternating current output.

IJISRT23FEB939 www.ijisrt.com 1459


Volume 8, Issue 2, February – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
 A power supply voltage range of 3.2 V to 12
 Excellent line and load regulation performance
 Current and thermal limiting
 Low noise; dropout detector
 low shutdown current of 1 A
 Thermally improved SOIC N package.

F. Raspberry Pi OS
All Raspberry Pi models can run the official operating
system, Raspbian. The OS was copied to the SD card using
Win32DiskImager, and the installation was checked using
VNC viewer Operating the Raspberry Pi OS in Windows
OS required the use of the VNC viewer editor.

Fig 10 APR33A3 Voice Module APR33A3 Voice


Module Function

 Fig 10 is the diagram of the APR33A3 Voice module


APR33A3 voice module Function and the pin diagram
as follows:

 VCC
This pin is responsible for powering the module. It
should be powered by a 5V supply.

 GND
This pin is used to connect the module to the ground. It
should be connected to the circuit's ground.

 REC Fig 11 VNC Viewer


This pin is used to activate the module's recording
function. The module will begin recording the voice The user was able to access and operate the Raspberry
message when this pin is pulled low. Pi's desktop environment from their Windows OS computer
using VNC viewer, as seen in Figure 4. This highlights the
 PLAYE versatility and flexibility of the Raspberry Pi, which can be
This pin is used to activate the module's playback used in various applications and environments.
function. When this pin is pulled low, the module begins
playing back the previously recorded voice message. V. RESULT ANALYSIS

 ERASE The project hardware is illustrated in Fig 12 This


This pin is used to delete the previously recorded voice project consists of 16 inputs and respective outputs. The IR
message. The module will erase the previously recorded sensors are connected to the hand glove.IR sensors detect
message if this pin is pulled low. the movement of our hands. As the movement of our hand
changes, there is a change in resistance which gives
 RXD and TXD respective voltage. Raspberry Pi Pico contains the memory
of different commands. The input command given is
These pins are used to communicate serially with the
compared with memory in Raspberry Pi Pico. Based on the
module. They can be used to program and control the
input HC-05 Bluetooth module will give appropriate output.
module's functions via a microcontroller or other serial
PCB is worked as a voice module. The command retrieved
device.
from Raspberry Pi Pico is sent to PCB which acts as a voice
module. This voice is converted into text by a Bluetooth
 Features of APR33A3 Voice moduleAPR33A3 voice-
text-to-speech converter app on mobile. There are 5 IR
module Function:
sensors each sensor connected to each finger. When a
 Exceptionally low dropout voltage: 180 mV at 200 mA gesture is given the appropriate output will be produced. In
(typical); table 5.1, the outputs for the specific inputs are shown.
Gesture 0000 gives the output “Good Morning”, gesture
 High precision over line and load: 0.8% at
0001 gives the output “Good Day” and gesture 0010 gives
 +25 °C, 1.4% over temperature
the output “Good Afternoon”.
 For stability, only 0.47 °F of CO is required.
 All capacitor types are stable with any cap (including
MLCC).

IJISRT23FEB939 www.ijisrt.com 1460


Volume 8, Issue 2, February – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165

Fig 12 Electronic speaking device Fig 14 Result for input command ‘1110’

Table 1 shows the gesture recounted output in the In Fig 14 bending the thumb is shown as a hand
sentences corresponding to the given input handgesture in gesture, and "come fast" is the output message.
the form of binary code.
VI. ADVANTAGES
Table 1 Result Table
Sl.Hand
No. Gesture (Binary Code)
Gesture RecountedOutput  The communication between a normal person and a
0 0000 Good morning speech-impaired person becomes easier.
1 0001 Good day  As we are using the display to show the user command
2 0010 Good afternoon so one speech-impaired person can also communicate
3 0011 Good evening with a deaf person.
4 0100 Good night  There is an option for user input.
5 0101 May I Come In  This device is portable and compact; the user can bring
6 0110 May I Drink someWater it anywhere he/she wants
7 0111 I want water
8 1000 I will come later VII. CONCLUSION AND FUTURE WORK
9 1001 I will do it later
10 1010 Well done The hardware protocol for speech-impaired individuals
11 1011 Let’s go utilizing hand gestures will be a useful tool and aid in
12 1100 Goodbye communication in workplaces and the public sector. By
13 1101 What is that? making it simple for people to communicate with them, the
14 1110 Come fast system will be able to successfully convert hand signals.
This project seeks to close the communication gap between
the quiet society and the rest of society. The gesture
symbol's potential to be an associated autonomous system
that applies to shared living areas is this project's most
noteworthy characteristic. Only a few commands can be
used with this system. The following is expandable in the
future to include more commands. The use of two gloves in
place of one is also possible with 16 modules.

Fig 13 Result For Input Command ‘0111’

The message "I want water" is represented by the


bendingindex finger in Fig 13.

IJISRT23FEB939 www.ijisrt.com 1461


Volume 8, Issue 2, February – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
REFERENCES [10]. K. S. Abhishek, L. C. F. Qubeley and D. Ho, "Glove-
based hand gesture recognition sign language
[1]. S. S. Kumar, R. Gatti, S. K. N. Kumar, N. Nataraja, translator using capacitive touch sensor," 2016 IEEE
R. P. Prasad, and T. Sarala, "Glove Based Deaf- International Conference on Electron Devices and
Dumb Sign Language Interpreter," 2021 International Solid-State Circuits (EDSSC), 2016, pp. 334-337,
Conference on Recent Trends on Electronics, doi: 10.1109/EDSSC.2016.7785276.
Information, Communication & Technology [11]. Sakshi Bajpai and D. Radha. “Smart Phone as a
(RTEICT), 2021, pp. 947-950, doi: Controlling Device for Smart Home using Speech
10.1109/RTEICT52294.2021.9573990. Recognition”. 2019 International Conference on
[2]. D. Hatibaruah, A. K. Talukdar, and K. Kumar Communication and Signal Processing, DOI:
Sarma, "A Static Hand Gesture Based Sign Language 10.1109/ICCSP.2019.8697923
Recognition System using Convolutional Neural
Networks," 2020 IEEE 17th India Council
International Conference (INDICON), 2020, pp. 1-
6,doi:10.1109/INDICON49873.2020.9342405.
[3]. L. Anusha and Y. U. Devi, "Implementation of
gesture-based voice and language translator for dumb
people," 2016 International Conference on
Communication and Electronics Systems (ICCES),
2016, pp. 1-4, doi: 10.1109/CESYS.2016.7889990.
[4]. C. Chansri, J. Srinonchat, E. G. Lim and K. L.
Man, "Low-Cost Hand Gesture Control in Complex
Environment Using Raspberry Pi," 2019 International
SoC Design Conference (ISOCC), 2019, pp. 186-187,
doi: 10.1109/ISOCC47750.2019.9027669.
[5]. X. Chu, J. Liu and S. Shimamoto, "A Sensor-Based
Hand Gesture Recognition System for Japanese Sign
Language," 2021 IEEE 3rd Global Conference on
Life Sciences and Technologies (LifeTech), 2021,
pp. 311-312, doi:
10.1109/LifeTech52111.2021.9391981.
[6]. A. K. Panda, R. Chakravarty, and S. Moulik, "Hand
Gesture Recognition using Flex Sensor and Machine
Learning Algorithms," 2020 IEEE-EMBS Conference
on Biomedical Engineering and Sciences (IECBES),
2021, pp. 449-453, doi:
10.1109/IECBES48179.2021.9398789.
[7]. L. Boppana, R. Ahamed, H. Rane, and R. K.
Kodali, "Assistive Sign Language Converter for Deaf
and Dumb," 2019 International Conference on
Internet of Things (iThings) and IEEE Green
Computing and Communications (GreenCom) and
IEEE Cyber, Physical and Social Computing
(CPSCom) and IEEE Smart Data (SmartData), 2019,
pp. 302-307, doi: 10.1109 /iThings /
GreenCom/CPSCom/SmartData.2019.00071.
[8]. C. R. P. Dionisio and R. M. Cesar, "A project for
hand gesture recognition," Proceedings 13th Brazilian
Symposium on Computer Graphics and Image
Processing (Cat. No.PR00878), 2000, pp. 345-, doi:
10.1109/SIBGRA.2000.895833.
[9]. H. Ishiyama and S. Kurabayashi, "Monochrome
glove: A robust real-time hand gesture recognition
method by using a fabric glove with the design of
structured markers," 2016 IEEE Virtual Reality
(VR), 2016, pp. 187-188, doi:
10.1109/VR.2016.7504716.

IJISRT23FEB939 www.ijisrt.com 1462

You might also like