Research Associate; Emote VR Voicer AI

The School of Digital Arts is a purpose built, interdisciplinary school at one of the UK's leading universities. Offering industry and research informed courses and specialist spaces with the latest technologies. The School of Digital Arts is a proud part of Manchester Metropolitan University. We build on the creative, science, tech and business strengths of a university whose research is rated as ‘world-leading' and is changing the way we live, work, learn and play.

AI systems are increasingly able to detect a speaker's emotions, leading to a new affective channel that can be explored in art. The controls available in standard Virtual Reality (VR) can be supplemented with speech recognition, natural language processing, and sentiment analysis. We aim to embody this potential in the front end of the Emote VR Voicer interface, which would translate detected meanings of vocal utterances to the morphing of abstract 3D animated shapes, enabling a radical new aesthetic experience.

We are using iterative design cycles and ultimately aim to develop an interface that will improve the participant's wellbeing. Candidates do not need prior arts-sector experience, only openness to interdisciplinary collaboration.

Working within the School of Digital Arts (SODA) you will join state-of-the-art research on the AHRC funded Emote VR Voicer project, to contribute to the development of an intelligently responsive VR app that incorporates speech recognition and meaning classification. You will be responsible for the AI development part, which will consist of making the system intelligent enough to listen to live mic input and classify to emotion (happy, sad, calm or angry) aiming for a success rate of approximately 80% or above.

In addition, you will make the model able to analyse language and classify to the same emotions based on that.

Key requirements for our system are minimum latency to allow the visual response to be immediate and a good success rate that will work on most visitors. You will research, use and fine‑tune existing models that will work on spoken or sung voice input. Once your model works in Python you will re‑code it, potentially as a plugin, to be easily usable in Unity.

No previous art experience is required but previous experience of interdisciplinary working is welcomed.

About the role:

You will be working closely within a small project team consisting of artists, a psychologist and AI researchers in an iterative development cycle. Your AI programming skills will be utilised to train and fine tune two AI models, one able to detect and tag emotional meaning from audio, and one classifying emotion based on the spoken/sung words. The outputs of these will steer real‑time visuals based on four dominant emotions in the Unity Games engine.

You will also be involved in writing up the research for publication. You will gain co‑authorship on interdisciplinary publications, portfolio‑quality work and exposure to exhibitions.

The job will be for 2.5 days per week (0.5 FT) on a fixed‑term basis for a maximum of 6 months. The working pattern will be partially on‑campus and partially remote depending on project stage and flexible, or caring‑friendly arrangements are welcomed where possible.

About you

Key skills:

• An excellent understanding of and experience with programming using Python for machine learning, C# and C++

Essential skills and experience:

• A PhD in computer science, AI, software engineering or a similar technical field, or equivalent professional experience
• Experience with machine learning and AI
• Experience with writing and co‑writing research papers
• Experience with speech recognition (e.g. Whisper), sentiment analysis, or emotion classification.
• Experience with Image and/or audio‑based AI development and familiar with libraries such as:
  Sci Py, Tensor Flow, PyTorch, Hugging Face
• Experience with audio‑based projects
• Experience with real‑time system optimisation (e.g. low‑latency audio/visual feedback in VR)
• Experience with data backup systems
• Experience with working in interdisciplinary teams
• Excellent communication and interpersonal skills.
• Creative problem‑solving skills
• Self‑motivation and capable of…