An expensive grader can rapidly increase RFT costs; in one early experiment with 100 examples, the team spent about $20,000 on the grader alone.

“do it like less efficient but with like sft let's say you're using the open ai you know to do some supervised fine turning you'll probably have like you know maybe a few thousand examples the job takes a few hours it costs you like 100 bucks right with rft maybe you have like 100 examples the job takes a few days and it costs thousands of dollars one way to burn money really fast is to use a very very expensive grader so i think one of our first experiments we ran it was like only 100 examples right we quickly burned like immediately 20k”

“Okay, just on the greater alone.”