aimo-progress-prize
AI Mathematics Olympiad Solution
- Fine tuning DeepSeekMath Base 7B model to solve mathematical problems
- Train using two high-quality datasets of mathematical problems and solutions
- Solution candidates generated by self consistent decoding algorithm
- Use validation sets from AMC, AIME, and MATH to guide model selection
- Train models using open-source libraries TRL, PyTorch, vLLM, and DeepSpeed
- Model training is divided into two stages: CoT training and TIR training
Product Details
This GitHub repository contains training and inference code for replicating our winning solution in the AI Mathematical Olympiad (AIMO) Progress Award 1. Our solution consists of four main components: a formula for fine-tuning DeepSeekMath Base 7B to use Tool Integrated Reasoning (TIR) to solve mathematical problems; Two high-quality training datasets containing approximately one million mathematical problems and solutions; A self consistent decoding algorithm for generating solution candidates with code execution feedback (SC-TIR); Four carefully selected validation sets from AMC, AIME, and MATH were used to guide model selection and avoid overfitting to the public ranking list.
