Podcast
Questions and Answers
Why is pupil diameter measurement considered vital in behavioral research?
Why is pupil diameter measurement considered vital in behavioral research?
- It provides insights into physiological and psychological states, such as stress levels and cognitive load. (correct)
- It is essential for diagnosing vision-related ailments.
- It helps in determining the physical fitness of individuals.
- It allows for accurate calculation of a person's age.
What challenge does using webcams pose in the accurate measurement of pupil diameter, as mentioned in the abstract?
What challenge does using webcams pose in the accurate measurement of pupil diameter, as mentioned in the abstract?
- Webcams require specialized software for operation.
- Webcams do not provide color images, making analysis tricky.
- Webcams can be expensive for widespread use.
- Webcams typically have lower resolution compared to specialized pupillometry equipment. (correct)
In the context of pupil diameter estimation, what is the primary role of upscaling techniques?
In the context of pupil diameter estimation, what is the primary role of upscaling techniques?
- To convert color images to grayscale for easier analysis
- To enhance measurement precision by improving the resolution of eye images (correct)
- To correct distortions caused by webcam lenses
- To reduce the computational resources required for image processing
Which of the following is a key contribution of the thesis regarding accessibility and analysis options?
Which of the following is a key contribution of the thesis regarding accessibility and analysis options?
According to the 'Declaration of Authorship,' which of the following principles was confirmed?
According to the 'Declaration of Authorship,' which of the following principles was confirmed?
Which equipment, traditionally used for accurate pupillometry, is mentioned as requiring specialized features and being traditionally expensive?
Which equipment, traditionally used for accurate pupillometry, is mentioned as requiring specialized features and being traditionally expensive?
In data collection for EyeDentify, what was the role of ChameleonView?
In data collection for EyeDentify, what was the role of ChameleonView?
Why was it important to change the background color of the computer screen during data collection?
Why was it important to change the background color of the computer screen during data collection?
To prepare the EyeDentify dataset, what steps were taken after aligning recordings?
To prepare the EyeDentify dataset, what steps were taken after aligning recordings?
What is Eye Aspect Ratio (EAR) used for in the data preprocessing pipeline?
What is Eye Aspect Ratio (EAR) used for in the data preprocessing pipeline?
How did the researchers address the challenge posed by lower-quality images from webcams?
How did the researchers address the challenge posed by lower-quality images from webcams?
Among the SR models used, which were specifically designed for face restoration and enhancement?
Among the SR models used, which were specifically designed for face restoration and enhancement?
What was the benefit of applying SR before blink detection, rather than after?
What was the benefit of applying SR before blink detection, rather than after?
What type of data was collected, in addition to pupil diameter and webcam images?
What type of data was collected, in addition to pupil diameter and webcam images?
Which data augmentation techniques were applied to prepare the datasets?
Which data augmentation techniques were applied to prepare the datasets?
What is the primary goal of integrating image SR into the pupil diameter estimation workflow?
What is the primary goal of integrating image SR into the pupil diameter estimation workflow?
Aside from improving diagnosis of medical conditions, which use case is mentioned for gaze and pupil diameter estimation applications?
Aside from improving diagnosis of medical conditions, which use case is mentioned for gaze and pupil diameter estimation applications?
Which condition is a limitation to prior work for measurement acquisition?
Which condition is a limitation to prior work for measurement acquisition?
What is emphasized to improve ground truth values for future works?
What is emphasized to improve ground truth values for future works?
What data is presented for end users in the final interactive format?
What data is presented for end users in the final interactive format?
What do users rate for the overall experience of pupilSense?
What do users rate for the overall experience of pupilSense?
What does Pupil Sense ensure, to protect the privacy of end users?
What does Pupil Sense ensure, to protect the privacy of end users?
Which of the following statements best describes a limitation of the EyeDentify dataset?
Which of the following statements best describes a limitation of the EyeDentify dataset?
Regarding enhancing the model precision, what can be implemented to create iris segments?
Regarding enhancing the model precision, what can be implemented to create iris segments?
For real-time capabilities in the web application, what models could integrated to be more efficient in those lower resources?
For real-time capabilities in the web application, what models could integrated to be more efficient in those lower resources?
Which areas are identified for future development within PupilSense, the web application?
Which areas are identified for future development within PupilSense, the web application?
What technique does not mention to improve features on the webcam imagery?
What technique does not mention to improve features on the webcam imagery?
What technology was used in the app to create the data?
What technology was used in the app to create the data?
What is one demographic details of the participants in the study?
What is one demographic details of the participants in the study?
What was used to optimize learning?
What was used to optimize learning?
To have a more diverse study, how can it be modified?
To have a more diverse study, how can it be modified?
How does integrating personalized profiles with large language models affect results?
How does integrating personalized profiles with large language models affect results?
Which type of study was done to analyze the validity of the app?
Which type of study was done to analyze the validity of the app?
Where does the web app launch?
Where does the web app launch?
Does the creative commons liscense used prevent what activity?
Does the creative commons liscense used prevent what activity?
Which of the following was not a recommendation from survey feedback?
Which of the following was not a recommendation from survey feedback?
What can researchers further accomplish in future studies?
What can researchers further accomplish in future studies?
What type of environment would enhance prediction of pupil measurement and overall model performance?
What type of environment would enhance prediction of pupil measurement and overall model performance?
Flashcards
Pupil Diameter Estimation
Pupil Diameter Estimation
Estimating pupil diameter from webcam images using super-resolution techniques, making pupillometry more accessible.
Super-Resolution (SR)
Super-Resolution (SR)
Techniques to enhance image resolution beyond the sensor's physical limitations.
GFPGAN
GFPGAN
A type of AI model designed specifically to enhance facial features in images.
CodeFormer
CodeFormer
Signup and view all the flashcards
Leave One Participant Out Cross Validation (LOPOCV)
Leave One Participant Out Cross Validation (LOPOCV)
Signup and view all the flashcards
Eye Aspect Ratio (EAR)
Eye Aspect Ratio (EAR)
Signup and view all the flashcards
WebGazer
WebGazer
Signup and view all the flashcards
Pupil diameter
Pupil diameter
Signup and view all the flashcards
Mean Absolute Error (MAE)
Mean Absolute Error (MAE)
Signup and view all the flashcards
SearchGazer
SearchGazer
Signup and view all the flashcards
ChameleonView
ChameleonView
Signup and view all the flashcards
Data Preprocessing
Data Preprocessing
Signup and view all the flashcards
EyeDentify
EyeDentify
Signup and view all the flashcards
EyeDentify++
EyeDentify++
Signup and view all the flashcards
PupilSense
PupilSense
Signup and view all the flashcards
Study Notes
- Estimation of eye pupil diameters aids insights into psychological states
- Accurate pupil measurement is traditionally done via specialized eye trackers and glasses
- A new method has been created using inexpensive webcam images and super-resolution
- The method aims to make measuring eye pupils more accessible
- Upscaling techniques range from bicubic interpolation to CodeFormer
- Accuracy hinges on selection of upscaling methods and scaling factor
- An application provides visualization via heatmaps, graphs, and blink ratios
Declaration of Authorship
- States all work is the authors
- Cites prior work of others, all sources are given.
Acknowledgements
- Thanks go to Prof. Dr. Andreas Dengel
- Thanks go to Brian Moser and Ko Watanabe
Measuring pupil diameter
- Measures stress levels
- Gauges cognitive load
- Acts as a valuable tool in behavioral research
- Related to short and long term memory processes of the human brain
Neurological conditions
- Alzheimer's
- Schizophrenia
- Parkinson's
- Opioid use
- Mild cognitive impairment
Common eye monitoring methods
- Drowsiness detection via blink detection
- User interface optimization using gaze estimation
Key Takeaways
- Diverse eye colors, user effort
- Position constraints and technical skills needed limit broader use
- Applications and devices with pupil size estimation can advance analysis
Challenges
- Ground truth data collection
- Sufficient diversity in the dataset
- Pupil Diameter Prediction
- Image Scale for Prediction
- User-Friendly System Development
Contributions
- ChameleonView, assists research of pupil diameters. During recordings screen alters to guarantee variety
- EyeDentify: Ground truth measurements of pupil captured alongside face video, yielding data from 51 participants
- EyeDentify++: Applying image SR to enhance images scaled 2x and 4x
- Deep learning models were assessed, split and tested to estimate diameters of both eyes
- Web application PupilSense created for public use
Related work
Gaze estimation datasets
- Various gaze estimation, blink detection, as well as facial image data have been collected, these lack data on pupil diameters
Pupil diameter datasets
- Data was collected from children with ADHD, this data set included only basic sensor values.
- Videos were recorded under light variation, also only 16 subjects had their pupil diameter quantified equivalent to 5mm.
Pupil Diameter Estimation Methods
- BINOMAP method uses slave cameras as a binocular geometric constraint for gaze images.
- CNN can operate on mobile systems for real-time assessment.
- algorithm PuReST, rapidly tracks pupils via head mounted eye trackers
- Yolo algorithm is often implemented
Super-Resolution as Pre-Processing
- Enhances medical imaging
- Provides more detailed geography insights
- Improves smartphones and TVs
- Enhances Smart-phone resolution significantly
- Pre-trained SR models is similar to image and object recognition
Dataset creation
- Includes hardware, data collection procedure, along with SR dataset
- Tobii eye-tracker is used to gather pupil diameter
- Microsoft Surface Studio 1 used for face and video recordings
Data collection application
- Specialized web application known as ChameleonView
- User friendly, enables webcam-based data
- Color changes capture different diameter sizes of the pupil
Data Preprocessing
- Webcam videos are combined with eye-tracking data
- Alignment
- Cropping the eyes
Aligning the video and pupil Diameter data
- CSV files are generated from the eye-tracker
- Records Diameters from pupils as well as gaze positions
- Used timestamp CSV file for webcam recording information
- Frames extracted are aligned with data points
Cropping the eyes
- Involves cropping the eyes from each extracted frame
- Landmark localization is completed using mediapipe, identifies facial features such as the eyes, nose and mouth to map face
- Eliminates the the effects of blinking
Super resolution
- Super resolution enhances images by a factor of 2 or x4
- A range of image SR approaches were employed to improve the quality of eye images
- Creates HR versions of the Low Resolution webcam images
- Eye regions are cropped before blink detection.
SR Techniques
- SRResNet is an method inspired by ResNet
- CodeFormer is a model based on VQ-VAE
- Model testing involves models known as GFPGAN, HAT, and Real-ESRGAN
Dataset details
- 51 participants
- Includes mixed genders and various ages
- Consists of a final count of 212073, removing those winking
- Dataset is made fully available.
Dataset distribution
- Includes multi display colors
- A web application known as ChameleonView records synchronized webcam, with a Tobii eye-tracker aligning at 90hertz
Dataset variation
- Allows for spontaneous movement by participants
- Captures a variety of different lighting conditions
- Captures a range of physical characteristics between different people
Training and evaluation
- Deep learning was used to train and evaluate new models
- Models were trained across the no-SR data, 2x data and 4x data
Models details
- ResNet employed as a base model
- With each ResNet architecture, custom linear layer was added
Dataset splits
- Effective dataset splitting is required to ensure the reliability of such models
- Implemented two methods of validation and one cross validation
Training details
- Trained using both cross validation strategies from the ground up
- With a batch of 128 images from separate left and right eyes
- Utilized the AdamW Optimizer
Results overview
- Upscaling improves accuracy for prediction
- SR is also effective in boosting element detail
Results: Scale Sensitivity
- A complex relationship can be seen between the upscaling factor
- Upscaling factor and models to generalize that not all are straight cut with scale and SR methods
Results: Potential Overfitting
- Overfitting is possible depending on the SR approaches used
- Some test configurations did not perform as strongly when assessed, model becomes too similar to training visuals
Results: Best Models
- Upsampling is the solution with optimal results, works with both left and right eye estimations for models
- Quantitative data shown that pre-trained models as compared can assess pupil diameter and provide error scores based on the SR method employed
Quantitative results
- LOPOCV tested again with a resnet model and left and right eyes recorded separately
- Class Activation Maps( CAM) show a singular participant with distinct colors across datasets
- CAM visualizations tested, including how lighting can effect accuracy
PupilSense Web-app details
- Web app, which estimates pupil size from images
- It presents pupil measurements
- It gives comprehensive recording analysis as well as maps to show model influence in the output metrics
- Creates and includes line charts illustrating pupil sizes
User Experience Demographics
- The survey was split into sections with users and their experience in mind
Overall experience
- Focused on application function, design, usability
Technical features and trustworthiness
- Advanced capabilities, ability to integrate diagnostic data
User Demographics
- Gathers quality input regarding users overall reaction
Survey Results
- The survey was quite diverse in opinion, with various views of background knowledge on both eye tracking and Pupil metric study
- Most found it user friendly
- Processing times also are of note to users
- Blinks, model action was well classified in surveys
- App usage was given high scores overall
- Some results are skewed or unnecessary
Overal Insights from feedback
- Feedback suggest that PupilSense was well designed, its strengths reside with its upload media
- However, feedback showed that most wanted faster times as well more precise explanation of data
Limitations and Shortcomings
- Primarily recorded w/o eyeglasses or medical conditions
- Prediction challenges of the study include data regarding diverse groups and lighting changes
- The web app created has limited time ability while also being prediction based alone
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
