| Paper Abstract and Keywords |
| Presentation |
2026-02-17 09:40
Inference of Cognitive States through Large Language Models and Gaze Information
-- An Examination of Mind-Reading Technology --
Nayuta Tada (Kindai Univ.), Fumika Nakanishi, Minori Yamataka (ARIC, DENSO CORP.), Takeshi Kohama (Kindai Univ.) |
| Abstract |
(in Japanese) |
(See Japanese page) |
| (in English) |
This study aimed to develop gaze-based mind-reading technology by extending BLIP-2, a deep learning model for image-to-text generation, to integrate gaze information into its attention mechanism. The training data consisted of gaze data measured while participants observed images after being presented with questions. Additionally, to reduce data collection costs, we created a subjective evaluation dataset in which humans assessed the instruction content of existing data and attempted to use this dataset for training purposes. The results showed that the model trained on the measured data achieved favorable scores for the evaluation metrics. Furthermore, we confirmed that varying the input gaze data for a given image alters the generated text. Conversely, training with the subjective evaluation dataset led to early overfitting, highlighting the remaining challenges in the data construction method. The prediction accuracy obtained with the measured data suggests that gaze information contributes to the proposed model's text generation, indicating the potential for developing gaze-based mind-reading technology. |
| Keyword |
(in Japanese) |
(See Japanese page) |
| (in English) |
Large language model / Gaze information / Text generation / Mind-reading / / / / |
| Reference Info. |
ITE Tech. Rep., vol. 50, no. 4, HI2026-1, pp. 1-5, Feb. 2026. |
| Paper # |
HI2026-1 |
| Date of Issue |
2026-02-10 (HI) |
| ISSN |
Online edition: ISSN 2424-1970 |
| Download PDF |
|
| Conference Information |
| Committee |
IEICE-HIP HI VRPSY ASJ-H |
| Conference Date |
2026-02-17 - 2026-02-18 |
| Place (in Japanese) |
(See Japanese page) |
| Place (in English) |
|
| Topics (in Japanese) |
(See Japanese page) |
| Topics (in English) |
|
| Paper Information |
| Registration To |
HI |
| Conference Code |
2026-02-HIP-HI-VRPSY-H |
| Language |
Japanese |
| Title (in Japanese) |
(See Japanese page) |
| Sub Title (in Japanese) |
(See Japanese page) |
| Title (in English) |
Inference of Cognitive States through Large Language Models and Gaze Information |
| Sub Title (in English) |
An Examination of Mind-Reading Technology |
| Keyword(1) |
Large language model |
| Keyword(2) |
Gaze information |
| Keyword(3) |
Text generation |
| Keyword(4) |
Mind-reading |
| Keyword(5) |
|
| Keyword(6) |
|
| Keyword(7) |
|
| Keyword(8) |
|
| 1st Author's Name |
Nayuta Tada |
| 1st Author's Affiliation |
Kindai University (Kindai Univ.) |
| 2nd Author's Name |
Fumika Nakanishi |
| 2nd Author's Affiliation |
Advanced Research and Innovation Center, DENSO CORPORATION (ARIC, DENSO CORP.) |
| 3rd Author's Name |
Minori Yamataka |
| 3rd Author's Affiliation |
Advanced Research and Innovation Center, DENSO CORPORATION (ARIC, DENSO CORP.) |
| 4th Author's Name |
Takeshi Kohama |
| 4th Author's Affiliation |
Kindai University (Kindai Univ.) |
| 5th Author's Name |
|
| 5th Author's Affiliation |
() |
| 6th Author's Name |
|
| 6th Author's Affiliation |
() |
| 7th Author's Name |
|
| 7th Author's Affiliation |
() |
| 8th Author's Name |
|
| 8th Author's Affiliation |
() |
| 9th Author's Name |
|
| 9th Author's Affiliation |
() |
| 10th Author's Name |
|
| 10th Author's Affiliation |
() |
| 11th Author's Name |
|
| 11th Author's Affiliation |
() |
| 12th Author's Name |
|
| 12th Author's Affiliation |
() |
| 13th Author's Name |
|
| 13th Author's Affiliation |
() |
| 14th Author's Name |
|
| 14th Author's Affiliation |
() |
| 15th Author's Name |
|
| 15th Author's Affiliation |
() |
| 16th Author's Name |
|
| 16th Author's Affiliation |
() |
| 17th Author's Name |
|
| 17th Author's Affiliation |
() |
| 18th Author's Name |
|
| 18th Author's Affiliation |
() |
| 19th Author's Name |
|
| 19th Author's Affiliation |
() |
| 20th Author's Name |
|
| 20th Author's Affiliation |
() |
| 21st Author's Name |
|
| 21st Author's Affiliation |
() |
| 22nd Author's Name |
|
| 22nd Author's Affiliation |
() |
| 23rd Author's Name |
|
| 23rd Author's Affiliation |
() |
| 24th Author's Name |
|
| 24th Author's Affiliation |
() |
| 25th Author's Name |
|
| 25th Author's Affiliation |
() |
| 26th Author's Name |
/ / |
| 26th Author's Affiliation |
()
() |
| 27th Author's Name |
/ / |
| 27th Author's Affiliation |
()
() |
| 28th Author's Name |
/ / |
| 28th Author's Affiliation |
()
() |
| 29th Author's Name |
/ / |
| 29th Author's Affiliation |
()
() |
| 30th Author's Name |
/ / |
| 30th Author's Affiliation |
()
() |
| 31st Author's Name |
/ / |
| 31st Author's Affiliation |
()
() |
| 32nd Author's Name |
/ / |
| 32nd Author's Affiliation |
()
() |
| 33rd Author's Name |
/ / |
| 33rd Author's Affiliation |
()
() |
| 34th Author's Name |
/ / |
| 34th Author's Affiliation |
()
() |
| 35th Author's Name |
/ / |
| 35th Author's Affiliation |
()
() |
| 36th Author's Name |
/ / |
| 36th Author's Affiliation |
()
() |
| Speaker |
Author-1 |
| Date Time |
2026-02-17 09:40:00 |
| Presentation Time |
20 minutes |
| Registration for |
HI |
| Paper # |
HI2026-1 |
| Volume (vol) |
vol.50 |
| Number (no) |
no.4 |
| Page |
pp.1-5 |
| #Pages |
5 |
| Date of Issue |
2026-02-10 (HI) |