Title Prepoznavanje emocija utemeljeno na holografskim i topografskim mapama EEG značajki i dubokom učenju
Title (english) Emotion recognition based on holographic and topographic EEG feature maps and deep learning
Author Ante Topić MBZ: 42124
Mentor Mladen Russo https://orcid.org/0000-0002-9363-6723 (mentor)
Committee member Dinko Begušić (predsjednik povjerenstva)
Committee member Mladen Russo https://orcid.org/0000-0002-9363-6723 (član povjerenstva)
Committee member Nikola Rožić (član povjerenstva)
Committee member Maja Stella https://orcid.org/0000-0001-7893-6464 (član povjerenstva)
Granter University of Split Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture (Department of Electronics and Computing) Split
Defense date and country 2022-07-19, Croatia
Scientific / art field, discipline and subdiscipline TECHNICAL SCIENCES Electrical Engineering Telecommunications and Informatics
Universal decimal classification (UDC ) 621.3 - Electrical engineering
Abstract Emocije su ljudske reakcije na događaje i one utječu na cijelo tijelo. Važna funkcija za izradu sučelja između mozga i računala (BCI) je razvoj modela koji je u stanju prepoznati emocije iz elektroencefalografskih (EEG) signala. Izazovan je zadatak razviti inteligentni model koji može pružiti visoku točnost prepoznavanja emocija zbog prirode moždanih signala. EEG ima nestacionarna i nelinearna svojstva te sadrži značajnu količinu šuma uzrokovanu, primjerice, mišićnom aktivnošću, treptanjem, disanjem, otkucajima srca ili slabim kontaktom elektroda. Štoviše, kod snimanja EEG signala neinvazivnim nosivim uređajima često se koristi veliki broj elektroda, što povećava računsku složenost, dimenzionalnost EEG podataka i smanjuje razinu udobnosti ispitanika. U ovoj disertaciji se predlaže novi model za prepoznavanje emocija koji se temelji na izradi mapa značajki korištenjem topografskog i holografskog prikaza karakteristika EEG signala. Signali snimljeni elektrodama mjernog uređaja se dijele u podpojaseve primjenom diskretne valićne transformacije, a na svakom podpojasu se računaju karakteristike signala koje se mapiraju na standardni međunarodni sustav koji opisuje pozicije elektroda na glavi. Prikazom vrijednosti karakteristike signala na lokaciji elektrode se definira položaj točke u trodimenzionalnom prostoru u kojem se istraživanje karakteristika signala provodi u dva smjera. Za prvi se koristi računalno generirana holografija kojom se iz prostornih karakteristika signala izrađuju dvodimenzionalne mape značajki, dok se u drugom smjeru istraživanja vrijednosti karakteristika signala prikazuju topografskom mapom. Metode ReliefF i analiza susjednih komponenti su upotrijebljene u istraživanju odabira elektroda sa svrhom optimizacije i povećanja točnosti modela. Pristup dubokog učenja korištenjem konvolucijske neuronske mreže je iskorišten za izlučivanje obilježja s mapa značajki, a dobivene karakteristike od svake pojedine neuronske mreže se spajaju u matricu značajki te se potom klasificiraju. Prepoznavanje emocionalnih stanja je provedeno nad svim ispitanicima zajedno, ali i zasebno ovisno o spolu ispitanika, a novopredloženi model je verificiran na četiri javno dostupne baze podataka DEAP, DREAMER, AMIGOS i SEED. Demonstrirana je učinkovitost predloženog pristupa u usporedbi sa najsuvremenijim studijama u kojima autori koriste EEG signale za klasifikaciju ljudskih emocija u trodimenzionalnom prostoru. Eksperimentalni rezultati pokazuju da se predloženim pristupom može poboljšati stopa prepoznavanja emocija.
Abstract (english) Emotions are human reactions to events and they affect the whole body. An important function of creating a Brain-Computer Interface (BCI) device is the development of a model that is able to recognize emotions from electroencephalogram (EEG) signals. It is a challenging task to develop an intelligent framework that can provide high accuracy of emotion recognition due to the nature of brain signals. The EEG is non-stationary, non-linear, and contains a significant amount of noise caused by, for example, muscle activity, blinking, breathing, heart rate, or poor electrode contact. Moreover, when recording EEG signals with non-invasive wearable devices, a large number of electrodes are often used, which increases the computational complexity, dimensionality of EEG data, and reduces the level of comfort of the subjects. This dissertation proposes a new model for emotion recognition based on feature maps created using the topographic and holographic representation of EEG signal characteristics. The signals recorded by the electrodes of the measuring device are divided into subbands using discrete wavelet transform, and at each subband, the characteristics of the signals are calculated and mapped to a standard international system that describes the positions of the electrodes on the head. The position of the point in the three-dimensional space is defined by displaying the value of the signal characteristic at the electrode location. Investigation of signal characteristics in three-dimensional space is carried out in two directions. For the first, computer-generated holography is used, which creates two-dimensional feature maps from the spatial characteristics of the signal, while in the other direction of research, the values of the signal characteristics are shown by a topographic map. The ReliefF and Neighborhood Component Analysis methods were used in electrode selection research to optimize and increase model accuracy. The deep learning approach using a convolutional neural network was used to extract features from feature maps, and the characteristics obtained from each individual neural network are combined into a matrix of features and then classified. Recognition of emotional states was performed on all respondents together, but also separately depending on the gender of the respondents. The newly proposed model has been verified on four publicly available datasets: DEAP, DREAMER, AMIGOS, and SEED. The effectiveness of the proposed approach has been demonstrated compared to the state-of-the-art studies in which the authors use EEG signals to classify human emotions into three-dimensional emotional space. Experimental results show that the proposed approach can improve the rate of emotion recognition.
Keywords
Interakcija između čovjeka i računala (HCI)
sučelje između mozga i računala (BCI)
elektroencefalografija (EEG)
prepoznavanje emocija
ReliefF
analiza susjednih komponenti (NCA)
trodimenzionalni emocionalni model
duboko učenje
neuronske mreže
računalno generirana holografija (CGH)
Keywords (english)
Human Computer Interaction (HCI)
Brain-Computer Interface (BCI)
electroencephalogram (EEG)
emotion recognition
ReliefF
Neighborhood Component Analysis (NCA)
valence-arousal-dominance model
deep learning
neural networks
Computer-Generated Holography (CGH)
Language croatian
URN:NBN urn:nbn:hr:179:447749
Promotion 2022
Study programme Title: Electrical Engineering and Information Technology Study programme type: university Study level: postgraduate Academic / professional title: doktor/doktorica znanosti, područje tehničkih znanosti, polje elektrotehnika (doktor/doktorica znanosti, područje tehničkih znanosti, polje elektrotehnika)
Type of resource Text
Extent 112 str.
File origin Born digital
Access conditions Access restricted to authenticated users
Terms of use
Created on 2022-11-11 11:37:51