Updates
2025
[May 2025]
Reviewing for the 2nd Conference on Language Modelling COLM2025.
[April 2025]
Our paper Fine-Tuning Pretrained Models with NVIB for Improved Generalisation has been accepted to be presented at ICLR 2025, Singapore at the Workshop on Spurious Correlation and Shortcut Learning: Foundations and Solutions!
[March 2025]
Reviewing for the 63rd Annual Meeting of the Association for Computational Linguistics ACL2025
[February 2025]
I wrote my first blog post! It’s a reflection on my research internship at Amazon and living in Berlin. Interning at Amazon Berlin 2024: the Grind, Growth, and Gratitude
2024
[December 2024]
Completed my research internship at Amazon, Berlin. They are tackling ambitious, unsolved problems, building AI agents to assist software engineers. I worked in improving retrieval for code editing agents. Special thanks to Prabhu Teja and Giovanni Zappella!
[November 2024]
Presented at the 30 years of Neural Networks Festschrift for Dr. James Henderson. The talk was a light-hearted story of my PhD journey. See the slides.
[August 2024]
Our paper Nonparametric Variational Regularisation of Pretrained Transformers has been accepted to be presented at COLM 2024!
[July 2024]
Starting a research internship at Amazon, Berlin. I will be working in the space of retrieval augmented generation (RAG) and Large Language Models (LLMs) for code generation with Prabhu Teja and Giovanni Zappella
[April 2024]
Reviewing for the journal IEEE Transactions on Pattern Analysis and Machine Intelligence
[March 2024]
Presented a seminar to the Department of Statistical Sciences at the University of Cape Town. The talk was on the evolution of NLP, the attention mechanism and my PhD Research. See the slides.
[January 2024]
Reviewed for NAACL2024 as an emergency reviewer.
2023
[December 2023]
Our work on Learning to Abstract with Nonparametric Variational Information Bottleneck will be presented at the Black Box NLP Workshop EMNLP2023 in Singapore!
[December 2023]
Our paper Nonparametric Variational Regularisation of Pretrained Transformers is on ArXiv.
[October 2023]
Our short paper Learning to Abstract with Nonparametric Variational Information Bottleneck is accepted to findings of EMNLP 2023 in Singapore, (Paper) (Demo) (Poster) (Code)
[Autumn Semester 2023]
Teaching for PhD: Deep Learning for Natural Language Processing at EPFL and Masters: Natural Language Processing at UniDistance.
[August 2023]
Interviewed by the Bayes Newsletter of the South African Statistical Association (SASA) - The Journey from Undergraduate to PhD
[July 2023]
Joined the company Defiant for a week in Banff, Canada as an AI consultant.
[June 2023]
Attending the Generative Modeling Summer School GeMSS 2023 in Copenhagen, Denmark.
[May 2023]
Attending the 2023 ICLR conference in Kigali, Rwanda
[April 2023]
Our paper HyperMixer: An MLP-based Low Cost Alternative to Transformers is accepted to ACL 2023, (Paper) (Poster) (Code)
[January 2023]
Our paper A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck is accepted to ICLR 2023. (Paper) (Poster) (Code)
2022
[July 2022]
Our paper A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck is on Arxiv.
[May 2022]
Attended the 2022 ACL Conference in Dublin, Ireland.
[March 2022]
Our paper HyperMixer: An MLP-based Green AI Alternative to Transformers is on Arxiv.
[Spring Semester 2022]
Passed the course Human language technology: applications to information access. This course allowed for fine-tuning skills in NLP and provided a more holistic picture of current research space.
[February 2022]
Successfully passed my candidacy exam. I am officially a PhD candidate of the Electrical Engineering Department of EPFL.
2021
[September 2021]
Participated in the SciFilmIt Hackathon Geneva 2021. This let me work with a diverse team to bring a probability concept through a visual medium for the public.
[Autumn Semester 2021]
Passed the course Scientific programming for Engineers. This course provided fundamentals in code development and practical applications in C++.
[Autumn Semester 2021]
Passed the course Deep Learning For Natural Language Processing. This course gave good grounding in NLP and allowed for a group project which will hopefully lead to a publication.
[April 2021]
Virtually attended the 2021 EACL Conference in Kyiv, Ukraine.
[February 2021] Joined the Natural Language Understanding group under the supervision of Dr. James Henderson at the Idiap Research Institute as a research assistant.
[January 2021]
Completed my MSc in Statistics at the University of Cape Town. My MSc thesis: Modelling non-linearity in 3D shapes: A comparative study of Gaussian process morphable models and variational autoencoders for 3D shape data.