Lei Chen, Xuanle Zhao, Zhixiong Zeng†, Jing Huang, Liming Zheng, Yufeng Zhong, Lin Ma*

MSRL (Multimodal Structured Reinforcement Learning) is a reinforcement learning strategy designed to break through the SFT performance plateau in chart-to-code generation. As the first method applying multimodal structured rewards to this domain, MSRL addresses limitations in handling information-dense visual inputs through an innovative multi-granularity reward system. The approach combines rule-based textual rewards that validate code correctness from five key dimensions with visual rewards using a "render-and-compare" mechanism to assess structural similarity between generated and original charts. MSRL employs a two-stage curriculum learning strategy, training initially on textual rewards before incorporating visual signals. Experimental results show MSRL improves the high-level metrics by 6.2% and 9.9% on ChartMimic and ReachQA benchmarks respectively, marking the first time open-source models achieve competitive performance with advanced closed-source models in the chart domain.

📢 News and Updates

• 2026.01.27 Our MSRL is accepted by ICLR 2026! 🎉
• 2025.08.26 We upload our model weights MSRL and MSRL-SFT to HuggingFace.
• 2025.08.19 🔥🔥🔥 We release the technical report of MSRL at arXiv link.

🤗 Models

The MSRL-SFT employs Qwen2.5VL-7B-Instruct as the initial model and performs supervised fine-tuning with a 2.8M Chart2Code dataset. The MSRL builds upon the SFT model and undergoes two-stage RL training using a high-quality 33K Chart2Code dataset.

📊 Performance

🔍 Usage Example

Below is an example of how to use MSRL for chart-to-code generation:

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

# Load model
model_path = 'DocTron/MSRL'
# model_path = 'DocTron/MSRL-SFT'

# Load the model on the available device(s)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(model_path, torch_dtype="auto", device_map="cuda")

# Use the following instruction and pixel range by default
instruction = """
You are an expert Python developer who specializes in writing matplotlib code based on a given picture. I found a very nice picture in a STEM paper, but there is no corresponding source code available. I need your help to generate the Python code that can reproduce the picture based on the picture I provide.
Now, please give me the matplotlib code that reproduces the picture below, starting with "```python" and ending with "```".
"""

processor = AutoProcessor.from_pretrained(model_path, min_pixels=1280*28*28, max_pixels=16384*28*28)

# Prepare input with image and text
messages = [
    {"role": "user", "content": [
            {"type": "image", "image": "assets/example_case.jpg"},
            {"type": "text", "text": instruction},
        ]
    },
]

# Preparation for inference
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
images, videos = process_vision_info([messages])
inputs = processor(text=text, images=images, videos=videos, padding=True, return_tensors='pt')
inputs = inputs.to(model.device)

# Inference: Generation of the output
generated_ids = model.generate(**inputs, max_new_tokens=4096, top_p=1, temperature=0.1)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
]
generated_code = processor.tokenizer.batch_decode(
    generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
)[0]
print(generated_code)

📌 Acknowledgement

We sincerely appreciate LLaMA-Factory and MM-EUREKA for providing reference training framework.

📖 Citation

If you find this project useful, please feel free to leave a star and cite our paper:

@misc{chen2025breaking,
      title={Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation}, 
      author={Lei Chen and Xuanle Zhao and Zhixiong Zeng and Jing Huang and Liming Zheng and Yufeng Zhong and Lin Ma},
      year={2025},
      eprint={2508.13587},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2508.13587}, 
}