View Proposal
-
Proposer
-
Radu-Casian Mihailescu
-
Title
-
Enhancing Efficient Reasoning in Smaller Language Models through Distillation Techniques
-
Goal
-
-
Description
- Large Language Models (LLMs) have demonstrated impressive performance in complex reasoning tasks. However, their high computational cost and resource demands pose significant barriers to widespread deployment. Knowledge Distillation (KD) has emerged as a promising solution, transferring knowledge from large models to smaller, more efficient ones. Despite this, traditional KD methods often focus solely on final outputs and neglect the intermediate reasoning steps that contribute to an LLM’s performance.
Research Problem:
How can the reasoning capabilities of large language models be effectively transferred to smaller models in a resource-efficient manner, especially in scenarios with limited annotated data?
- Resources
-
-
Background
-
-
Url
-
-
Difficulty Level
-
Moderate
-
Ethical Approval
-
None
-
Number Of Students
-
1
-
Supervisor
-
Radu-Casian Mihailescu
-
Keywords
-
-
Degrees
-
Bachelor of Science in Computer Science
Bachelor of Science in Computer Systems
Bachelor of Science in Information Systems
Master of Engineering in Software Engineering
Master of Design in Games Design and Development
Master of Science in Artificial Intelligence
Master of Science in Artificial Intelligence with SMI
Master of Science in Business Information Management
Master of Science in Computer Science for Cyber Security
Master of Science in Computer Systems Management
Master of Science in Computing (2 Years)
Master of Science in Data Science
Master of Science in Human Robot Interaction
Master of Science in Information Technology (Business)
Master of Science in Information Technology (Software Systems)
Master of Science in Network Security
Master of Science in Robotics
Master of Science in Software Engineering
Bachelor of Science in Computing Science
Bachelor of Engineering in Robotics
Bachelor of Science in Computer Science (Cyber Security)
Master of Science in Robotics with Industrial Application
Postgraduate Diploma in Artificial Intelligence
Bachelor of Science in Statistical Data Science
BSc Data Sciences