Grant Sanderson 谈 AI 与数学的未来

0:00Today I'm chatting with Grant Sanderson who runs through Blue and Brown and is now working on a new project documenting the progress AI is making in math. And I wanted to talk to you about this because AI has been making the fastest progress in mathematics as of any other field.

0:13So whatever is happening here and whatever we're seeing AI progress happen or not happen would tell us about what will happen to the rest of the world as AI gets better and better. So I wanted to start with this question I asked you when I first interviewed you three years ago. And I asked you,

0:27once we have AIs that can get gold in the International Math Olympiad, wouldn't that just be AGI? Wouldn't this just be able to do anything any human can do, given how hard these problems are? And you had an answer, which in retrospect turned out to be very wise and correct, which is like,

0:40it'll be another benchmark, like all these other benchmarks that they are passing. Obviously, AI has gotten better in general ways since then, but there won't be some aha moment when this happens. First, I think I'd be curious to get your heuristics on why that turned out to be true.

0:56And second, I'm curious how long you think this narrowness can continue to be true. So by the point that AI has solved the million price problem, do you think it's still possible that at that point there's lots of tasks that humans are doing that AI still can't automate in the economy?

1:11It's an interesting question because it's hard to answer without knowing what the solution looks like ahead of time. I mean, if we take the IMO, that's something where I think the spirit of your question three years ago was in looking at how some of the solutions to these problems really seem to require creativity.

1:26And the designers of these problems, they'll try to have them come up with things that you can't train for as easily. I think the dirty secret with the IMO is that you really can train for a lot of them. And so... With the whole AI and math project undergoing, I think, as you point out,

1:41one of the reasons it's interesting at all is that there's a spiky frontier to AI. Math is just right there in one of the spikes. But there's kind of a fractal nature to that spikiness because when you zoom into the specific progress within math, you have some things that are a lot easier than others.

1:56So if we just think about IMO, which is old news at this point, it's kind of like two years ago, they're really doing quite well. They would have gotten a gold in 2024 if for not the following reason. They're very good. They're just like cold solved geometry, basically. And the IMO has these four categories of problems.

2:12That's geometry, number theory, algebra, and combinatorics. So like geometry just solves in like 19 seconds in 2024 because it's kind of a brute force solver. And the dirty secret is for students, there's also sort of a brute force way that you kind of can go at it.

2:26Combinatorics is the one that's the wild card of much more like playful, puzzly seeming problems. And there were two combinatorics problems on that year's test. There's not always. There's four categories, six different problems. So it's kind of a toss-up, which one is going to have two questions.

2:43Had it been more geometry questions, they would have gotten a gold that year. But it struggles on those combinatorics ones. And, you know, someone who's trying to keep that torch of the last holdout of, like, math for humanity might say, well, you know, those are the ones that require the more creativity. Even then, though, I think...

3:01The spirit of your question on, like, if they're solving, you know, a Millennium Prize problem, does that also service a lot of white-collar work? It suggests that whatever the rate limiter is between where we are now and that is the same as the rate limiter for making things better at white-collar work. And we can maybe, like,

3:17paint a couple different ways that, like, we focus on, I don't know, Riemann hypothesis. Like, what would it look like to solve that? One possibility would be these things are extremely good at a specific domain of knowledge and just knowing it very deeply and then knowing another domain and knowing another domain

3:35and you've pointed this out it's like bizarre to have something with this superhuman breadth that like knows all the field so well that's not just finding those lightning bolts that connect them I think we're starting I'm starting to see sparks of that, of like actually finding connection between the things that it's an expert at.

3:50I'm sure we'll talk about it. If the nature of the solution to the Riemann hypothesis was something like that, that feels pretty distinct to me than what's necessary to get good at white collar work. And there's a reason to believe actually that that might be the nature of the solution.

4:04I don't know if you know the story of like Hugh Montgomery and Freeman Dyson at the IAS. This is a side tangent, but it's just kind of a fun story on how... I don't know if it was over lunch or something like that. Basically, you have this number theorist who is pointing out just trying to understand

4:20the statistical correlation between pairs of zeros of the Riemann zeta function. So the Riemann hypothesis is all about like, do all these zeros sit on a straight line? It's like this quantitative question you could ask about. And he writes down a formula. It looks like one over sine squared or something like that.

4:35Freeman Dyson, a physicist, is like, I know that expression. That expression comes up in studying the eigenvalues for random Hermitian matrices, which was something that comes up in studying the energy levels of like a nucleus. And the idea that the statistics of those two seemingly different things were the same sort of prompted a potential exploration on hey,

4:55are there aspects of random matrix theory that might be relevant to Riemann zeta function. And I think it's a little bit of an open question, like is there food to be had there? But that kind of bridging together from two different fields, like if it turned out that the solution to the Riemann hypothesis was exploring

5:11an idea like that even further, that has this character of kind of how you expect LLMs to be good at math. It's like they're an expert at the quantum physics, they're an expert at the analytic number theory. They should be able to see that similarity in a way

5:23that doesn't require like Montgomery and Dyson to be having lunch and like happening to talk about that. That's totally different from white collar work, right? In terms of like the extent to which you maybe have a hard time using an AI as an editor. It's not because they know everything and you just need them to find

5:39that lightning bolt in between. Different possibility would be... What's the right analogy? Maybe like if we think of Fermat's last theorem between the moment of Fermat phrasing the question and then what the solution itself looks like, where ultimately the solution involves such heavy machinery in math, right?

5:57So the beauty of that problem is you can phrase it so simply. You ask about, you know, x to the n plus y to the n equals z to the n. Do you have integer solutions for this when n is bigger than 3? And it's... It's something you might expect there to be an elementary number theory approach

6:12to it, but just as far as we can tell, there's just not. Whereas the actual solution, you know, maybe there is something simpler, but this might be what it has to be. There's such a complicated set of ideas that build on like centuries of work centered around elliptic curves.

6:28And then this other like mountain of ideas centered around these things called modular forms. And like both of those mountains have to be built before you can ask the right question that connects it. So if the solution to the Riemann hypothesis involved building a new mountain, like that's a kind of skill,

6:42like the ability to like come up with the right new ideas that feels sufficiently different from like the character of how they're intelligent right now that... It's not like that's what you need from your hired video editor per se, but that if it's capable of building mountains that are the correct new theory

7:00that crystallizes how we should be thinking about a subject, that's just such a level of intelligence that then it starts to feel Like it would be surprising if that didn't permeate into other aspects of the economy besides like just the mountain building for math itself.

7:13Yeah. Or at the very least even if it couldn't like literally do every single thing white collar humans can do. Yeah. It would just have transformative effects in the way that getting gold in the IMO did not have transformative effects on the world. First of all,

7:25I do want to point out that I'm totally moving the goalposts here because when I interviewed Dario about two or three years ago, I asked this question about why haven't they been able to use their vast knowledge to connect ideas together and come up with a new discovery that way. That seems like the kind of thing,

7:38even if a moderately intelligent person knew this much information, they'd be able to like come up with a medical diagnosis from the fact that like this drug causes migraines and this other thing, you know, whatever does this and maybe it's the same drug that can cure both things. And yeah, I don't know, from an outsider's perspective,

7:53mathematics seems clearly like a field where finding this counterexample to the unit distance problem conjecture was like an example of this kind of thing. As a total goalpost moving, but then we can ask, okay, what is the next benchmark now that AIs can do this thing that we should have thought they should be able to do.

8:09What is the next thing that would be quite impressive? And there's a couple of candidate ideas here. So one could be coming up with interesting problems in the first place. And the other is coming up with new kinds of objects or conceptualizations that create or unify fields. On the first one,

8:27right now we just train these models to, like we have these Millennium Prize problems because Riemann came up with this idea of this like Riemann's data function and because he thought that it would have some connection with like the density of prime numbers or if the zeros on this function would have some connection

8:43to prime numbers and so like figuring out that there's why do we think this is an interesting thing to study in the first place why are we building this object and trying to answer questions about it and answer this particular question about it seems like the kind of thing that would be the next benchmark

8:59I mean, you highlight two pretty good examples there. For anyone curious about the unit distance conjecture, there's this really nice video by a math channel called Polylog where they talk about it. And one of the people in that, because all of these discussions, it causes people to reflect on like the process of doing math, right?

9:15They're like, ah, this thing can do this impressive stuff. Like, what does that mean for us? And he highlights this quote, how good mathematicians prove theorems, great mathematicians come up with conjectures, and the greatest mathematicians come up with definitions. And that's more or less exactly your framing here on like those two, like we need the conjecture generator,

9:33and then like the definition generator, that's the premium tier mathematician. I don't understand how exactly you'd make that a benchmark in the sense that usually when I think of the word benchmark I'm thinking something that you have like it's a goalpost that the ball is through the goal

9:48or it's not like you can clearly say like yes this is done partly to be able to do things like our LVR but also partly just to be able to like know that you haven't moved the goalpost and answering you know OpenAI can have their headline on disproving

10:00the unit distance conjecture because it's a clear distinct it's like it did it right whereas imagine trying to have a headline on like 255.4 came up with a really good conjecture. We promise everyone thinks it's a good conjecture. It just doesn't land the same way.

10:16But maybe that doesn't negate the fact that that's the right thing to be thinking about. So I would be surprised if it ever took the form of looking like a benchmark and we have a score saying that it's past this benchmark because we can quantify how good a conjecture it is.

10:30But probably the nature of what it would take is that You would feel a tone shift in conversations with mathematicians about the way that it's useful to work with, right? And like this series that you referenced that is not at all produced yet

10:43and probably won't be for a couple months takes the form of us interviewing a lot of mathematicians And what's interesting is we started doing this like over a year ago. And it's fun to see a little bit of a tone shift in the way that they talk about AI

10:54between like mid 2025 and where we are now in 2026. You know, in the real world, that's a very short amount of time. In the AI world, that's eons. and we're able to see over those eons this tone shift. I think the way that you'd measure conjecture generating ability is going

11:10to be more subjective on that tone shift where it'll be mathematicians saying they're not just using it to solve their problems, but as they step back and decide what their research field should even be, that a conversation with such and such model was genuinely helpful for that.

11:24I don't think it's likely that you'd see it in the form of a headline saying that this was yet another benchmark knocked down.

11:32Right, and so it's very interesting, the kinds of things you can't make benchmarks for are also the kinds of things, at least in the current paradigm, you can't easily train for, right? Because there's really no fundamental difference between a benchmark and a training environment. I think it's very easy to come up with some dichotomy of like,

11:50here's a deep reason why AI can't do a certain thing and then it turns out, well, you're just thinking about it the wrong way and actually I can do it pretty soon thereafter. But I'm going to come up with... You're going to come up with a couple anyway? And I think that this will probably,

12:04it'll probably turn out that there's ways in which we can train AIs to do these kinds of things in the relatively near term. But it seems like it would have to be different from current RLVR training. So the thing I'm curious about,

12:13and the thing it seems to me that drives a lot of the big progress in mathematics and science generally, is like coming up with a new way to think about a problem or the new way to understand the world that then... unifies different fields, spawns entire new fields,

12:30solves problems we weren't even thinking we were trying to solve in the first place. The reason Einstein was thinking about GR is not because he wanted to explain why light bends or why black holes exist. These are phenomena he didn't even know it needed to be explained in the first place. But in mathematics, it often seems...

12:46Okay, a total outsider, I don't even know the details of what I'm talking about here. From the outside, it seems like there's often ways to... say prove a specific problem that can motivate a new conceptualization uh one which results in a whole new field the whole new way of thinking which is immensely productive and one which doesn't

13:04i think um i'd be curious to hear you talk about whether uh galwa coming up with group theory distinguishing his like solution to the The Quintic having no formula for the roots and Abel coming up with a different proof a few years earlier that didn't come up with group theory but then

13:18if you wanted to do a verification loop on like is group theory an interesting concept that was like was something useful done here why is this proof better potentially that verification loop is a hundred years long and it involves the cryptography coming around and physics making progress and the ideas

13:33in group theory being relevant and understanding like symmetries in physics and all those kinds of things it's like a hundred year verification loop but why is this a productive concept in the first place

13:41Yeah, boy, yeah, you struck a nerve because I had this like project about Galois I was going to do in 2022 that I put on the shelf, but I spent like a year of my life like thinking a lot about what he did. So there's a risk of me accidentally talking too long on the specifics

13:56that you can hold me back on. It's a perfect example for your case because Describing why it was a valuable insight does not come from immediate utility. And so certainly if you're thinking about RLVR environments, it's like, okay, this is going to be really hard to do. But it's interesting to note how even with like human

14:17The human verifiers at the time, it took a really long time to recognize it as being useful. I think Einstein with GR, people sort of felt, you can feel this feels like a good theory right away. What makes the Galois theory such an interesting example is you have literally this 100-year segment of an idea

14:34that flows through many different people's heads before it settles into something that the math community agrees is good. To back up a little bit, I mean, do you want the background on the problem at all? All right. Well, so we all learn about the quadratic formula in school.

14:50I thought you were going to say, we all learn about group theory in school. I miss that class. We all learn about group theory, about quadratic formula.

14:57So this was known in some sense, like Greeks could solve quadratics, but they didn't really write things in algebra. And so it's really more like the Arabs that like wrote down like that formula. There's this delightful story around some like... dueling Italian mathematicians, not real duels, just like intellectual challenges who like secretively found a formula for

15:18the cubic. And then very shortly thereafter found a formula for degree four polynomials. So a natural open question and for like mathematicians is, can you find a formula that solves degree five equations? Now the degree four, it's monsters. It's like, it would be wild to write it down. You usually don't really write it down in full.

15:36You break it up as like a procedural thing. So you might believe these things have this exponentially increasing complexity. So many hundreds of years, nobody is like really answering that question. Usually we say Abel was the first to prove it. He was this young, precocious Norwegian mathematician, and he showed it's simply impossible.

15:54It's not that you can find a quintic formula. He thought he found one, but he showed it's impossible. I think the real credit though, like you have to back up a little bit and talk about Lagrange, where Lagrange found the right kind of question to ask about this.

16:08I can go into the details if you want, but I'll give it a very high level. He was studying the question and he recognized being able to solve these polynomials is actually very related to understanding like the way that certain algebraic expressions are like symmetric, like more or less so.

16:24Like if I write down a plus b plus c plus d, just like adding four variables. it doesn't change the value of the expression whereas if i write like a plus b multiplied by c plus d some other permutations don't change it but some of them do and he had this really

16:38nice insight about how if you can find expressions like this that have four free variables That had this unexpected relationship with being able to reduce degree four into degree three. So he started approaching the, like, can we find a quintic polynomial by saying, I wonder if I can extend that. And to extend that method,

16:57you would have to have an expression that has five free variables such that as you permute them over all the five factorial permutations, it takes on only four values or fewer. So you could put that in a puzzle book. You could put that in a brain teaser that a 12-year-old could engage with.

17:12And it's not too hard to find yourself feeling like that's an impossible task. And so Lagrange is sitting here saying, hmm, here's a strategy that I'm trying to solve this problem. Can I find a quintic polynomial? This strategy doesn't. It seems like it might be impossible, at least from this strategy.

17:28But that was the first time in history that people had the instinct that some kind of question about symmetry was the right way to be studying these polynomials. In his mind, it was just a way. It had yet to be discovered that actually there's a tighter connection. And also, maybe rather than searching for the formula,

17:43we should be asking the opposite question can you prove that it's impossible so he sort of planted that seed like around 50 years later Abel definitely read Lagrange and was influenced by it Galois we know that he loved Lagrange when he was like falling in love with math and so it's very hard to imagine

17:57that like these two young geniuses the fact that they both come up with like pretty similar insights around that problem it's not like born from Lagrange but to your question on like are you are you able to verify that this was a good idea there were There wasn't any like result that Lagrange came to.

18:12There's never like he solved the problem and therefore we know that that was like the right question to ask. He asked it. There's some like intrinsically interesting thing. It also wasn't very important for math at the time. Like most people were more interested in like the applications to physics. this is almost in

18:26that like side almost recreational hobbyist type thing like Abel you know he started working on Quintic stuff but then he was advised to spend more of his efforts studying elliptic functions and so more of his work was on that before he died young he died at 26 from tuberculosis and then Gawa he he

18:46pushed both of those ideas in the right direction where he really understood the nature of abstraction. And so he had this really nice piece that he wrote while he was in prison, actually. He was like, we could talk all about his life story. It's pretty wild. But he's like this teenager. He's in prison.

19:01He had tried to submit his math papers and they had been rejected So again it's like verifiable reward The like verifier function that is the academy at that time is rejecting what he wrote Because frankly it was not very coherent Like it wasn't a complete proof He wasn't giving like a clear thought of like what

19:15the theory actually was He was just like a young fledgling mathematician getting his bearings So it's like the verified reward there is like no good But he has some instinct that there's something there So he's writing this diatribe on like the nature of like math being something which is um

19:30It undergoes these like shifts over time and he talks about like the advent of just algebra itself and going from just thinking in terms of numbers to like having a certain fluency just with like pure algebraic expressions where you're not tied to interpreting those expressions and he has this instinct that like there is another layer of abstraction

19:48that seems like what we should be doing where rather than thinking about the formulas themselves thinking about like what symmetries underlie those formulas But it was still a pretty like ill-defined theory. So if you're trying to say, okay, is the verified reward that like he has solved a problem that other people haven't?

20:03It's like, well, Abel proved that Quintics are unsolvable. And you say, what was Galois doing? Well, in principle, the thing that Galois theory will let you do is take a specific policy And it gives you the rules to say does that specific polynomial have roots that you could write down? For example,

20:16like x to the fifth minus one, you know that a solution is one, or x to the fifth minus two, you can write down fifth root of two. So it's not that every quintic polynomial you can't write down the solution, but could you find a specific one where you prove you can't write the solution using radicals?

20:30He also didn't even solve that exactly.

20:32Like he has a much more abstract, he didn't show for a specific example that he couldn't. So even describing like what problem did he solve is very tricky. So then he dies. It's this very like romantic story of he has this duel. We can get more into it.

20:45There's a lot of myth around like supposedly he writes up all his ideas the night before the duel. Really he tried to get them published like five times before.

20:50Working in the Quintic doesn't seem to be good for your health.

20:52It's very bad. Yeah, yeah, yeah. If you're a young genius, don't work on the Quintic. And so he asks his brother and his close friend, like, get these notes to Gauss, get these notes to, like, the important mathematicians of the day because I think there's something here. Even then, it didn't really take,

21:06like, so his brother and his friend, like, tried to get them out. It wasn't another 20 years until Louisville, like, sees these notes, sees that maybe there's something in them and tries to, like, clean it up and understand, like, what was Galois getting at. And then even then,

21:19it was another 20 years or so until Jordan actually, like, puts together a... Something like a modern treatment of group theory that they attributed to Galois. You could easily imagine history turning differently where like these ideas were kind of coming about from other points in math and like Galois could have been forgotten in history if he was

21:37a less like florid character. But between the time of Lagrange, like having this inkling of maybe symmetries of roots is the right way to go to where it all looks like modern group theory. Like you've got this long span. A lot of the time, it's like not even passing the like verified reward of human reviewers, right?

21:54Because it like gets on someone's desk. They say, I don't really know if there's anything here. It gets on someone's desk. They don't. You have to have this like one person sort of recognizes it. And then even then, it's not really solving practical problems at that point. Like you point out cryptography and physics and things like that.

22:07You have to get into the 20th century before you have like Guelman thinking, hmm, maybe understanding the nature of like how certain groups like breakdown has this relationship with what particles are made out of. And like he anticipates quarks based on a purely group theoretic question.

22:25And like that's one of the more interesting applications of group theory is that like to even predict the existence of quarks is a group theoretic-like question. That's so long after Lagrange before you have anything like that. And so You have to ask, what is the way of measuring progress that's not based on solving a problem, right?

22:44And that's somehow capturing, what is the instinct that's inside Galois's mind when he says, I think there's something here? What's the instinct that's inside Lagrange's mind when he says, I think this is the right way to think about it? What's the instinct inside Louisville's mind when he says,

22:57these scattered notes from this long dead youngster might have something to them? It's so hard to put a finger on that, but I mean a different like series of videos I'm making right now is about like the whole compression is intelligence idea and even though this isn't really

23:13the angle I'm taking you know there is something to the idea that the smaller expression that's more predictive like feels more intelligent and so I wonder the extent to which you can give some kind of Verifiable reward around not just like did you solve it or what is it solving,

23:29but around the smallness of the concepts required to do it. I mean, going back to Riemann hypothesis solutions, what would that look like if an AI solves it? I think a third way that it could happen is it just straight up the same way that you could maybe have an elementary proof of fermat's last theorem

23:44that's just like spelled out over like thousands of pages that would be incoherent but like the cleaner way to view it is with elliptic curves and all that maybe there's some like thousand page proof of remand hypothesis that's like not no one's really getting anything out of it and what you actually want is like

23:58what are the succincts like compressed versions of those ideas like they would then lend themselves to human understanding like i don't know komagorov complexity like maybe you throw that into your like Your attempt to quantify what you mean by elegance. But I don't think it's easy,

24:15but I do think it's something you would have to do in order to reward the Galois-like instinct rather than just rewarding have you solved a problem.

24:23It's very hard to come up with the heuristic for science. But it's clear like humans have been doing this somehow and like obviously AIs will do it at some point Well it's relevant also not just

24:34in terms of verified reward but like presumably the end goal is understanding like human understanding and so even if you do have some like thousand page proof of some math thing or some like grand new physical theory the goal is understanding Yeah Right Maybe if the goal is predictiveness you can just have like

24:50Automated engineers go off and build rocket ships or something. We're like, we have no idea how these work, but we can get between stars. But there's going to be a lot of people who want to understand. You're still going to want whatever the concision function is that distills down,

25:03here's this complicated way of thinking into the right one, like the equivalent of the universal law of gravitation for Newton. You would still want to train AIs to be able to do that and find the compressed representation.

26:15the AIs will prove their human hypothesis and our understanding of mathematics won't be any the better for it. I have a couple of questions about this. The first one is whether this is like a thing you should expect. Like isn't the reason humans come up with general natural objects and sub goals

26:34and whatever when we're working on a big problem is that it's just like when you're trying to work on a complicated important problem and so we can just think about theoretically would this even be a simpler way to solve the Rayman hypothesis as opposed

26:46to just coming up with the natural abstractions that are relevant to thinking about the problem and then two empirically is this what we observe when AIs do make progress on problems today when the um when the ai came up with that counter example to the unit distance problem conjecture

27:00you can just read its chain of thought and it seems it's not understandable to me because I don't know anything about mathematics but it seems to other mathematicians it was like understandable and it made it made use of like known concepts of mathematics and like proved relationships between them

27:12and all the natural language and as a result accelerated our understanding of the connection between this object and this conjecture so is this even like empirically is this a thing we should be worried

27:23about I think it depends on the nature of Yeah, like again, if we sort of break down like the three possible ways of like solving the Riemann hypothesis, that one and the other like big one from this year was like a certain Erdos problem numbered like 1196, but it's about these things called primitive sets.

27:40But basically, it had that character of bringing an idea from a seemingly different field. As soon as you just present the basic idea to a mathematician, you say like, hmm. What if we like use this like try the Markov chain process where we show

27:54that this thing is one from the bottom up probabilistically rather than the top down and like use the von Mangold function. If you like say that to someone in the know they'd be like they'd kind of know how to run with it. So you have this very like small idea that has the form of expertise

28:06in one field expertise in another draw a little lightning bolt between them. Like those are Those are going to be very human parsable, right? Because all you have to do is just like show the start and end point of what those connections are. If the character of it is mountain building,

28:19you do have to put in a lot more time to like understand that new mountain that was built because it's like a new thread that's not just like lightning bolt between them. And if the nature of the progress was just like raw hustle, right? It's just like this super long thing,

28:32no new theories, but it's just like long, long, long chain of reasoning answer. then you would have that where it's like okay there's this whole digestion process so I don't think there's one clear answer I think it depends on what the solution there would look like and on the mountain building side

28:46I would actually be really interesting to see like is it by default a very human understandable like the way that we like see new theories from like great mathematicians or is it like a like an alien different kind of mountain being built where we even have to like reprocess the kinds

29:00of abstractions that we we engage with right well the closest example here would be like the you know the attempted solution of the ABC conjecture that was we maybe shouldn't get into that one but that it It's probably not a correct solution, but basically it's this whole new way of thinking

29:18that this otherwise reputable mathematician in Japan had come up with. And it just took mathematicians a long, long time to even parse what he was saying, but it had the feeling of just like an alien bit of mathematics that's theory building. It's not just like long-term

29:32and so the fear that you would have is that like yeah it does that the biggest fear would be that it does that and then much like the abc conjecture people work for years to go up the mountain this just isn't right if it turns out to be wrong but it really looked right

29:49even if it was right there's a lot of effort to like hike up a new mountain yeah

29:54If we end up in that situation, David Bessis had a really great blog post called The Fall of the Theorem Economy, where he's talking about this, you know, historically, as you were saying, mathematics is coming in about these definitions and problems, and it's about proving theorems about them.

30:12And that really the theorem proving stuff is what gets all the credit, but it's like really a parasite on the definition stuff. And historically, it's not even a problem in terms of credit apportionment, because If you come up with a definition, you're probably going to be the guy who comes up with a theorem.

30:27But now we're in a situation where if the valuable work is coming up with the insight, and then AI just automates the latter part. So, okay, imagine a scenario where we have AI comes up with like the Abel-like direct arguments about a bunch of important conjectures in the world, and then we just have these proofs.

30:46And now it's up to humans or to future AIs to then consolidate I mean, I'm sure if you had access to it, it would make it easier for you to then think about like, well, what is going on here? Is there some deeper way in which you can understand why this proof works

31:07that would make it easier to come up with the ideas behind group theory?

31:10Yeah, I think it would... It would be hugely helpful, right? Because so much of trying to discover new math is mostly being wrong. You're trying to solve a problem. It doesn't feel like constantly taking the correct step up the mountain. Mostly it feels like a random drunken walk where you're doing a thing

31:29So if at the very least you know that trying to digest what you know is ultimately leading to like a correct solution, like that feels like progress simply because it's providing like a sense of knowing that it leads to a solution. And there's plenty of like instances in the recent history of math

31:46where it feels like the reach has sort of exceeded the grasp where there's things that are proven like long before they're understood. And I mean, one of my favorite like openings to a paper it's not even like a research paper is more like an expository

31:59one is from this um mathematician named timothy chow who was trying to understand the concept called forcing and so there's this problem called the continuum hypothesis that more or less asks um like You have a size of infinity for the natural numbers. You have a size of infinity for the real numbers. Is there something in between?

32:16And the answer is both yes and no. It depends on your axioms. It's sort of outside the scope of our usual axiom systems, which is an interesting answer. But the method to describe it is just really, really hard to understand. It's the thing called forcing. And in the beginning of this paper,

32:30he writes, like, everyone knows the idea of an unsolved research problem. Like, I want to propose the idea of an unsolved expository problem where, like, I'm sure we've proven it but we don't really know why it's true and suddenly he proposes like a partial solution to that expository problem you can imagine why I loved that framing

32:47because like this is my whole life it's like I don't do research math it's just it's just wholly about like what's the most clear way to understand this even if it's proven just like there's a difference between proof and explanation and so on that side I think that You are basically getting to the importance of that distinction.

33:04Yeah. And that will be the main incentive for, or the incentive would have to change in not just mathematics, but in other areas of science from proving things about the world to consolidating proofs into problems or higher level insights. But we were having a discussion earlier at lunch about a recent talk you were giving

33:25about design and how it helps us understand things. And then In the limit, is there really a difference between the conceptualization for an idea and the idea itself? So, you know, if you think about special relativity and like space-time diagrams and Minskowski's space-time, is it like, yeah, this is like a way in which we illustrate this idea

33:47of like why there's length contraction and time dilation. But is that like, is it like that is the reality? So the exposition does seem to be like the explanation in some sense here.

33:58Yeah, I mean, there's a couple interesting things there. One is, it seems like there's a really strong correlation between the people who come up with genuinely novel insights and also are actually quite clear in their communication of it. Like, you might imagine, given that the experience of a university student is often that

34:14the expert there teaching them is not necessarily the best explainer of that topic because they are so spoiled by their expertise. But what seems, at least in some cases, to be the case is how... The people who are really coming up with something quite novel, so you've got like Einstein or like Claude Shannon,

34:29something there, you read their papers, they're really lucid papers, right? It doesn't feel like, oh, this is just for the experts and you have to chop through it with a machete to get, they're like very good expositors, like Feynman has this characteristic too, like very good expositor. And so maybe...

34:44The same part of the brain that comes up with the correct new way of thinking about it at a research level also has this knack for like good explanation. And I think this is pertinent to the AI one where I kind of used to think that AIs will become these automated theorem provers,

34:59but like the role of the mathematicians is going to shift towards like my job, like explain these things. I kind of suspect that actually they'll also be like quite good at doing that and probably just like better than most humans are at like doing the explanation half and distilling half.

35:14And that's actually not what's left for the mathematicians is like digesting and explaining what was going on. Probably the nature of how these things are going. I could envision, we can talk about like ways this might not be it, but like probably the same thing that is coming up with like

35:28the really good new idea that solves some new problem is just also good at explaining it. Yeah. That's my new, like that's a way my, I think, beliefs have changed.

35:37What's the last thing you think you will be doing? Both you and then also what would the mathematical community, the human mathematical community will be doing?

35:45I will probably be doing something like what I am until I die. Even, so like, even... I know the doomers are right.

35:54Maybe that'll be the same. Exactly. It'll be for the same reason.

35:58Yeah, yeah. You know, it's, you're like, build a man a fire and he's warm for one night, but set a man on fire and he's warm for the rest of his life. So that's where I am with AI. No, I, because some of the

36:10Some of the like function of an explainer or a teacher is to like add clarity to a thing that someone's curious about. That's one thing. But some of it is like a little bit more relational and a little bit more like providing like motivation, providing a sense of curation. One interesting take that I've heard

36:26about what mathematicians will end up being is actually more analogous to art museum curators than anything else, where they solve the things, so the art exists, right? They even know how to explain it really well, you know, all there. You still want someone to help you navigate in this nearly infinite space

36:43of what ideas are worth engaging with, like someone kind of doing that. And that one, even if AIs were in some sense better at that, I think we would always still prefer a human that we had a relationship with because the way that we get motivated to be interested in things is a social phenomenon.

37:00If you have some specific technology you're trying to build, that might be different. You need to know there. the people listening to this podcast they sort of trust your curation on like what's an interesting topic in the first place it's not that they're landing on here because whatever your next

37:13topic is that's like what they in a prior sense wanted to understand they're trusting you as yeah so my role and arguably that of like other mathematicians might actually just shift subtly into that curation direction of what ideas are worth displaying and that's a lot

37:28of my job right now even now it's basically like i think people think a lot of the time for a video goes into the visuals like sure a little it is not like immediate but like actually a lot of it is just deciding what's worth saying in the first place

37:41or what's what's worth putting there um and because that is That's just I want to engage with that and I think I have a trust with certain people and they are curious what I would choose to put forward even if the AIs are better than that in the same way

37:54that like human musicians are always going to have a role because of that like social function of the story behind them even if the like objective quality of the mp3 file coming out is like better from some model that's kind of what I see happening to my job yeah

38:08I want to go back to this question of earlier I was we were sort of just as AI has crossed this threshold this important benchmark of being able to connect existing ideas to come up with a new discovery or prove or disprove something just as it's crossed

38:23the threshold we're like okay but what's the next thing um i wanna just um there's

38:27a lot more to do on that one by the way like just because a couple lightning bolts have been started i still i think there's like this flourishing future over the next couple years of like really connecting yeah and so

38:36in the limit you could even say um i don't know if this is accurate to say it but potentially A lot of maybe the biggest breakthroughs look like this at some level. It's just general relativity. Oh, you're just connecting together Ramanian geometry and special relativity, right?

38:54And so as AIs keep getting better and better at this connection thing, maybe a lot of big breakthroughs are not really of a different qualitative nature. I don't know if you have a take on that.

39:04I mean, a lot of the conversation focus has been on problem solving and that nature of math, you know, like taking off Erdos problems or something. I would say it's not even a majority of mathematicians who would maybe characterize their work as like really targeting the next problem to take down.

39:18Are you familiar with like the Langlands program? No. Ah, okay. It's not even a field of math so much it is like a research ethos where Fermat's last theorem is one inkling of this on you had like these two different seemingly disparate things and a connection between them like led to a solution. Langlands was a mathematician.

39:39He has this like famous letter now, essentially spelling out how it seems likely that there's a lot more connections like that. And even got like a little bit more specific about the nature of the connections such that you might imagine this like large map and you've got this like valley and this like set of planes over there

39:56And there's a lot of mathematicians who would characterize their work as being part of like trying to understand the threads Like on this map And the progress there, it's not even like here's this one specific problem that we know will be solved by that connection It's more that there's been enough time -and -time again cases

40:11where big problems were knocked down By finding connections that it's almost preemptively finding the connections And so you could have Yeah it's actually very interesting that like this um Any time you run into a mathematician, ask them whether the character of their work is more akin to Langland's program or

40:30if it's more akin to targeting one particular problem, and you get a certain bifurcated split there. But the possibility of AIs being supercharged connectors feels like it might be an amplifying tool in that pursuit. Hard to measure though, right? Because this cuts to what we were saying earlier.

40:51How do you assign a score to say, yes, you've done it? If it's knocking down a problem, you have a clear way of saying, yes, you've done it. You can write the headline. You can have your PR move as the AI company to say we did it.

41:03Whereas if it feels like that was the right connection drawn, you can write theorems around it. And this is the nature of what the papers in that field look like. But I think it... I think it will require a lot more like human in the loop

41:14to basically like say what was it like the kind of connection that we're going for but that's my guess on what most of the useful progress from these models will look like like in the next five years is just really filling in that landscape of like connections that you can draw if you're an expert

41:30in multiple fields like you've pointed out it's kind of surprising we haven't already had this right and what I'd be curious like I would be curious to know at a technical level What causes the unlock there? Because on the one end, you can kind of paint an explanation in your head for why you could be an expert

41:48in all of these things and not be drawing those connections, which is when the thing is reasoning, like the method of reasoning is this autoregressive chain of thought phenomenon. Autoregression is actually like a really, really weird way to produce stuff, I think, if you think about it. You're an intelligent person.

42:07Imagine I've locked you in a box, right? And then the only way that you have of interacting with the world is that you receive a slip of paper and then someone says, can you like predict what will come next, right? And then you predict what will come next and then your memory is wiped, right?

42:20And then you get like another Clip of paper and you go, imagine that was done a whole bunch and then what comes out on the other end, they're like, look at this essay that you wrote. You might look at that and be like, this is awful. That's not the essay that I would have written, right?

42:31Because like the process of like repeatedly like predicting something is just pretty different from how you would think as a writer to like compose it and think it through and everything. And in particular, what would probably happen is you're sort of a slave to your context where you you might be answering some question about some particular field

42:48as we like drawn all the context around that and you're going there the the connection that actually is where all the substance is going to come from is like by its nature a very like unlikely one and you know you can do all

42:59the RL that you want to try to like get better in some way but like what's the thing that's specifically up weighting and incentivizing making these unlikely connections when the vast majority of them like aren't the predictable you know next token that would come in there and so it's like

43:13it might be the case that you just have this intelligence that's sort of locked in there inside that box but it's just a weird way of interacting with it so the thing I'm curious about is like do you ever get any fruit by just like

43:25Questioning the premise of how tokens are generated like every now and then in some way, right? And I don't think it would be as simple as you like manipulate the temperature or something like that. But like are there any things that you can do that take like the existing level

43:38of intelligence but like find the right ways of sparking those connections that like unlocks these sorts of things that we're seeing? Or do you need just a little bit more intelligence such that at the level of prediction, it's kind of predicting that it should be making that lightning bolt to another field.

43:55I think it's more productive to reason instead of architecture or even loss function to reason about data. Like, I don't know, we have diffusion models that do text and they're like, not out of the kinds of things that produce are not a wholly different character they're just not been explored as much i think

44:14the more relevant thing is what is the data on which whatever architecture whatever loss function you have is incentivizing you to um uh produce and um It does seem like they're getting better at like, okay, forget about math. I mean, we did have this, a couple of examples of this kind of thing.

44:32But if you just look at why are they getting better at being autonomous agents, it just, I don't know, they have like, they're in an environment where auto-regressively producing the step that says, let's step back and do a search over the whole code base. Right. And then let's step back and like assess my mistake.

44:48It's like the thing that works. I assume what happened in the case of... progress in science or maybe in math is you have frontier math like problems which require like mathematicians to specifically design them because they require connecting together two different fields and there's all I'm guessing there's all kinds

45:07of clever like partially synthetic ways in which to make harder and harder problems like that that require these kinds of connections um for example by like eliminating assumptions and still requiring the AI to continue to uh get to the answer um and then like it just doesn't really end up mattering what

45:22the loss function is it just like it's really about can you come up with an environment which incentivizes this ability yeah it feels like you should be able

45:30to I can't I certainly can't speak to the correct ways of doing that that like unlock all this but it would just be pretty surprising like don't you think it would be kind of surprising if over the next three years there's not just like a lot more

45:42of those lightning bolts so this I think is an important thing to think about which is we often think about how smart a single system is and we don't think

45:49about AIs having advantages that are more the result of other facts about them so In this context, the key fact about them is that we can just parallelize and arbitrarily scale them so that whatever level of capability they have, it's not just like one idiosyncratic genius in the history of mathematics who makes

46:11a few connections and then dies in a duel. I feel like this is among the many advantages that digital minds inherently have that we don't think enough about the fact that you can the other ones being the fact that you can like they can merge all the knowledge together at least

46:31that there will be techniques that allow this to happen that you can like that you can spawn off copies with identical levels of knowledge But yeah, I feel like this parallelization is like quite an important property. And I'd be curious about your predictions of even if they're not as smart as your mathematicians,

46:46the fact that they are just,

46:47you know,

46:47billions of because for PR reasons that the AI companies are just dumping billions and billions of dollars at this would have a quantity has a quality all of its own.

46:57If that seems in the right direction, I think, I mean, if we take that, you know, that conversation between Montgomery and Dyson at the IAS that like suggests some connection between Riemann hypothesis or Riemann zeta function zeros and random matrices, that feels like the kind of thing that you could try to like automate and that you have,

47:17you know, agents representing expertise in all these and basically having, okay, We all know that an institute is smarter than an individual and that like the reason for having people all in the same geographic location is because you want those like serendipitous conversations to happen. What does it look like to sort of engineer those between agents?

47:35I mean, it's interesting You sort of point out like you can sort of pool all your knowledge. I actually wonder if one of the advantages is that you can do the opposite of that where you have sometimes when an AI is failing it's because it sort of gets into

47:48a bad chain of thought and it's really hard to get it out of it. I just like start again same deal with humans right like sometimes you like start thinking about it in a certain way and actually what's required is to just like back up maybe sometimes the form of

48:01that you know their stories about people trying to prove something for a long time and then at some point they say hang on a second What if I tried to prove that it's impossible, like prove the opposite? And that like unwinding your own context and going at it with a fresh mind, you could imagine systematizing that

48:15or like having multiple different agents deliberately given different pieces of context and try to like comparing trust there. Like we don't have the same level of manipulation on our own context. In this like AI and math series, the first episode will be about like when they solve the IMO.

48:31And I want to focus on one specific IMO problem that they failed on, which is one that a lot of very smart students failed on. Terry Tao also failed on it. And the nature of it is basically that People were very mad at the problem because they called it a troll problem.

48:47I almost don't want to spoil it because I want to construct the episode around leading someone in without knowing that it turns out to have a simple solution because you can really empathize with what it's like to be a student solving this. Basically,

49:02there's a really elegant way of going down what you really feel like is going to be the solution based on the context of being the International Math Olympiad problem, positioned as it is, the character of the solution is really enticing, but it's kind of hard to prove that it's the best. The reason is that it's not.

49:16There's this almost brain-dead solution that is the best. And so the relevance of that to the whole AI story is for a human, What's required to answer that question is to escape your context. Escape the context that you're in the IMO. Escape the context of the way you've been trained to solve these contest math problems.

49:34And if you just approached it like a brain teaser that I throw someone off the street, they'd probably answer it well. And you sort of want the same sometimes for... Like human research in other contexts where like sometimes just being able to say refresh your thinking come at it completely differently so of all

49:52the advantages that digital minds have that might actually be one of them like a little bit more of a systematic what does it look like to like refresh your thinking try answering two separate questions like spin off two agents one who's trying to prove it one who's trying to disprove it one

50:05who tries it like this way and one who tries and they like deliberately have different contexts I would be curious to see if we're having this conversation three years from now how many of the like significant results that make headlines have that character of basically like erasing the context previously like trying a bunch

50:21of different things as opposed to merging the results of like a bunch of different

50:25It is incredibly interesting because a common concern people have about AIs is this entropy collapse where they all think the same way because they're trained in similar ways. This is why they're bad at writing. They kind of just like go down the same path and have similar patterns of speaking and so forth.

50:40But maybe actually the key advantage AIs have is that you can systematically... It sounded like one of the reasons the unit distance problem conjecture took so long to be disproven was because people assumed the conjecture was actually true. So mostly they were trying to figure out ways in which to prove it.

50:57And so maybe one of the key advantages the AIs will have is actually to... Increase the entropy by systematically trying out both the negation and trying to prove the positive of any given statement or being able to systematically give different agents different biases.

51:16That's a good point.

51:17It seems like an important thing in the history of human science is that Einstein is just really motivated by this bias that things should look the same in different reference frames. And then he had multiple other biases like this. But that is just very formative in his thinking. And you can just like Yeah.

51:36And so you would suggest basically like systematically increasing entropy at the prompt level even though you have this like inevitable collapse at the like auto regression level. Yeah. Yeah. And I mean Einstein would be an interesting example because it's like he's got this bias towards things to be able to do. He also has a bias towards

51:54like God should not play dice, right? And it's almost like you want to make sure that you don't accidentally have all of your LLMs or Einstein because you might halt on quantum mechanics progress,

52:03right? Which actually goes to show you that there's not a correct heuristic for science. Exactly. You actually just need multiple independent research Yeah, yeah.

52:11And that feels like old school software, right? As long as you're able to describe that in some way, you have old school software that amplifies that entropy in some way. And if you're able to put a clear ontology to the distinct ways of thinking that you want to prompt, you explore that full ontology.

52:28And then each individual one runs off doing what it is. I think there's a certain design question there on how exactly do you describe the different approaches. The easy one is, are you trying to prove it or disprove it? The harder one would be to say, what are all the tactics that you could take to prove this?

52:44And make sure that you're applying sufficient breadth to exploring that.

52:50I don't think people appreciate the kinds of things that these models can just go handle for you when you equip them with a good harness like cursor. For example, I started publishing my episodes on Bilibili for a hopefully burgeoning Chinese audience, but everything I upload there needs sponso'd segments cut out.

53:06Normally that would've meant I would have to ask my editors go back through all the old episodes, cutout Obviously, AI for math is making a lot faster progress than everything else, and people point to verifiability of the domain as the key reason this is happening. I think that's one of the two important reasons,

53:59but I don't think, I think people really neglect the other one. And I'm outside the labs, I don't know what's actually going on, but this is a totally naive theory. Okay, a tangential question to why AI is making so much progress in math. Why has it been so slow to computer use?

54:18which is what you, you know, computers is actually very verifiable. It's like, you know, is my Etsy package coming? Or like, is my event booked? You know, whatever. These are extremely verifiable things to survey. What computer use lacks is grindability. So because websites have like bot detectors and also it takes a tremendous amount

54:36of compute to run parallel rollouts. It's very hard to just run like a thousand parallel rollouts at the same checkout flow on Amazon because you'll get like shut down by Andy Jassy, right? Personally. Presses the red X on Dorkesh button. Exactly. And so you can try to build clones every single website.

54:55This is very labor intensive and slows you down. And the reason, by the way, you need to do so many parallel roles in order to learn a skill currently with deep learning is that we haven't solved sample efficiency.

55:07Sucking supervision to a straw.

55:09Exactly. Of course, people are working on many different techniques, but fundamentally, there's this big problem in There's this big constraint in the way you train AI that we just with code also you can containerize a given level of progress in a repository and then just spin out thousands of parallel containers or hundreds

55:27of parallel containers and say like try to implement this feature and It's totally deterministic. And because it's deterministic, you can solve the credit assignment problem because you know that whatever caused this rollout to succeed and this one to fail, the diff is the thing that like worked. And this way you solve the credit assignment problem.

55:41If you have situations that are starting off at different starting points, this credit assignment problem becomes much harder to solve. But most of the things in the real world are just very hard to containerize in the same way. Like coding and math are... exceptions to this rule.

55:55But if you're just trying to figure out how do I build a new business that succeeds? How do I like go trade in the markets for a day and like make money? You can't like the fact that you had to interact with the real world and like things change day after day means that you can't keep replaying

56:08and grinding and farming the simulator. But the math of course is the exception and I feel like this is actually an important driver of progress in this domain and also in coding. It's not just verifiability. It has to be grindable. The third reason that people point out that AI is making fast progress is they focus

56:28a lot on lean and formalization. Again, I have literally no idea what's going on in the labs. I feel like lean just doesn't matter that much for like the current level of progress in AI or like why is AI able to solve the unit distance problem? Well, they, or sorry, disprove the conjecture by the unit distance problem.

56:43They released the chain of thought or at least a rewrite of the chain of thought. Didn't have any lean in it. I think it just like the process-based supervision that lean provides where you know each step is correct seems like less relevant than just having this grindable outcome that is verifiable.

56:59It's an interesting point like grindability mattering more I guess I will say on the yeah okay so naively you might think lean provides something unique for math because you're able to see if it can prove it you have old school software that can tell you yes or no you use that as your VR I mean what

57:17so what would corroborate your point is the idea that like the initial attempts again I'll just circle back to IMO it's like initially DeepMind basically does that it's like everything in lean and then the next year it's all in natural language so to your point not needed I do I think there is a um

57:32a yet to be explored benefit of that formalization domain which is at the moment you still need you know ultimately like a human is reviewing that counter example to the unit distance conjecture to say looks good and that That provides a certain bound on how endlessly explorable things are. If you consider AlphaGo,

57:52AlphaZero style stuff where they're just off in their own universe, just playing a bunch of Go and exploring themselves, just completely going potentially off the rails of what any human needs to that they still have this automated verifiable reward. It's not just that, hey, you can do RL on that. It's also,

58:10you basically never have to check in and you can just like poor compute at them like exploring the universe of Go. What stands to be interesting, like maybe this won't pan out, but I think the jury should still be out on like whether this will yield anything. With Lean, you could imagine having a,

58:28basically endlessly running program that's constantly trying to extend mathlib. So mathlib, it's this GitHub repository that's basically like all of math written in code. It's very far from all of math, but they want it to be all of math written in code that you can ask like, is this proof correct? It's very labor-intensive to write these proofs.

58:45There's like a whole sub-community around it. But you could imagine, what if you just had an AI where you say, simply try to extend Mathlib. Maybe it's a fork of it so that it doesn't have, you know, like trash in it because people, you know, people

58:59So you have like your fork of like the pure AI mathlib and it just goes and just like doesn't stop. It doesn't need anybody to check in on it, right? It could just keep going. It might come up with its own conjectures. It might come up with its own theories and like different definitions.

59:14maybe many of them are useless but it just has this infinite tree that it can like grow out that's a very unique thing that math has that nothing else has where you could press go and then just like just just pour compute at it and like look away for 10 years and then come back

59:27and say like what do you have and there's there's gonna be something right and then there's a question is it useful or not like how do you suss that out That's just an interesting thing to be able to do. It would be very surprising if that didn't yield like some sort of interesting mathematical insight from it, right?

59:41So I think like that's the real case for, okay, there's like two different ways that like Lean is important in this story. That's the first one that then basically is how it's like you could let go, not even check in, and progress will be made. You can do that with Go.

59:55I don't think you can do that with natural language math.

59:57Hmm. That's very interesting. Did you see Karpathy's auto-research idea? He wrote this basically one Python file that does basic LLM training and then just had a repo where agents would try to make modifications to the file. If it sped up the speedrun, the modification stays.

1:00:18Eric Jang who came on to explain how AlphaGo works did a similar thing when he was trying to build in a very strong GoBot and he had interesting observations about the kinds of like it's really good at just go running an experiment and going down that path but it's bad at stopping at dead ends

1:00:35and just doing extremely parallel things anyways this will probably be changed this will change in the future it's very interesting to think about What it looks like in the limit. I mean, this is fundamentally like what the human institution of mathematical research is, right? It's just like, this is a library extended in interesting and useful ways.

1:00:53And this way you don't have any outcome-based supervision. There's no outcome that you're trying to incentivize. But you have a process You know the steps are correct You just don't know if it's going in an interesting direction But yeah like if you were doing

1:01:05that You don't want to completely go off the rails And like do a random walk through the space of logic You'd probably want some like supervisor model That's trying to provide heuristics on whether it's useful or not But yeah something of that character I mean you know people are working on it And like

1:01:19That's one of those like five years from now I'd be curious to like be able to get the future version of us like talking about whether like maybe that goes nowhere but Terry Tao was talking about one like research project that's basically try to exhaustively search the space

1:01:34of possible like algebras like you could imagine different like axioms that you apply to algebraic systems and so like when we come up with group theory there's a certain axiom system that like has this flavor of they kind of look like arbitrary rules unless you know the motivation

1:01:47but it's basically like what if you tried all of them? Do any of these yield useful things? And the vast majority of them is just trash in some way. It all collapses to no interesting results. But every now and then there would be this little island of

1:01:58a completely different type of axiom system that at the very least seems rich in terms of the number of theorems that can come out of it. And that's like bread and butter for what you would imagine automated provers being good for. It's like exploring that space and seeing which one of them turns out to be something.

1:02:13Like maybe one of those islands actually turns out to be something you can retroactively put motivation on to say this is the kind of structure that's trying to get at in the same way that you could imagine looking at the axioms for a group not knowing that it's about symmetry

1:02:24but retroactively realizing like wow this is very relevant to studying symmetry. So you could imagine results of that flavor but instead of just exploring possible algebra systems it's like all possible like logical consequences of any kind of axiom.

1:02:39on the point about whether you can provide process-based supervision without lean so DeepSeek had their DeepSeek math model and they released a paper on how they trained it and it was quite interesting so they have the problem with having natural language proofs is you don't know if it's correct or not and

1:02:56so they have a verifier and then the verifier is trained by a meta-verifier that makes sure that all the problems that they're training this model to solve and like the art of problem solving that the verifier is giving good feedback on And so it's just interesting natural language verification with some sort

1:03:13of meta verification kind of work at least seems to work so far in the published literature. And also it seems to work in the published products that we're using. Like if you look at coding agents, They're getting better and better at like writing clean code and refactoring code and stuff like that.

1:03:27And I'm sure that there's process-based like LLMS judge kinds of things which are saying, trying to provide taste and say, hey, is this like a clean way to write this function? Are there duplicates of the same kind of modular forms and so forth? I feel like that should also work for mathematics, right?

1:03:46It seems more plausible for math than anything else, even if you're only working in natural language, that you could trust a verifier. I mean, you and I were talking earlier about why they're bad at writing, and I was asking why you can't just have ... They seem to be good judges.

1:04:00If I give them two essays that students write, they'd be able to say which one's more accurate and insightful. So why can't you just have a verifier saying, is this a good piece of writing or not? And maybe the ultimate failure there is even if they're good at discriminating between a B essay and an A essay,

1:04:17they're not actually good at discriminating between an A essay and a thing you actually want to read that would be followable on Substack and insightful and all of that. They actually end up preferring just uninsightful pieces of writing. and so on the math front I guess the question would be like that step

1:04:34to simply know like is this a correct proof or not that lends itself to like an automated verifier even in natural language you could probably still make a ton of the progress it still doesn't I still like the sort of tree of logic out of lean front just in that you can really go off the rails, right?

1:04:50Like there's just no constraint on like the previous way that things had been phrased before in the same way that, you know, everyone talks about like move 37 in like AlphaGo and such. like what is the thing that lends itself to just going outside the prior heuristics

1:05:05and it seems productive to have a disconnection from the rest of the world in that exploration as like a complementary research pursuit to the natural language math front I mean the other relevance of Lean there would be like okay let's say you have your pure natural language language

1:05:24RL environments and you have a pure natural language set of proofs and people have to said like proceed AI mathematicians and they go and they generate like 10 papers a day that produce a bunch of stuff if the error rate if there's like any error rate to that at all So Alex Kontorovich has talked about this.

1:05:43It becomes insufferable, like as a mathematician, because you would basically be like, every single time I see one of these, I kind of don't know if it's worth my time. Even if 99 out of 100 of them are right, I don't know if it's worth my time to even go through it.

1:05:55because it's really labor -intensive to find what that error would be and it's like really frustrating if turns out you spent all your time on a paper that was trashed and so having anything that's able to give you that green track mark that says even if this is going to be complicated to understand

1:06:09even if it's going to Be A Pain, you at the very least know it is correct. Like every other field would kill for that right? And like math has that um if if the models are also able to take their natural language proofs and formalize them and so that seems huge right? The ability to have that

1:06:24every field would love to have something like that and so i think you are right Yeah.

1:06:44I also love this extension of MathLiv as a metaphor for what's going to happen to our civilization pretty soon. For millennia, humanity is building this corpus of knowledge and understanding and everything that we have now distilled into these models. And at some point, the models will just extend that arbitrarily. By the way, on the writing front,

1:07:08I actually have a theory of why writing is making worse progress than these other domains. So I think one of them is what you said, that they're bad at judging not only A versus B, but they get like just totally derailed by B star which is this like shitty essay that just hits all

1:07:25the all the bells and whistles that like A is supposed to hit and then so the reward hack thing just like totally goes off the rails but I think the other important thing is that writing is not modular in the same way that code and math are like

1:07:40you know you can write a function many different ways and they kind of do the same thing and of course you want it to be very clean and stuff but like at the end of the day it works it works same with like lemmas and mathematics and

1:07:48then you know you can like have some end product that is different from the way it is produced so the code is the thing that produces some end product and you want a functional end product whereas in writing the end product is directly the thing the AI is producing and each Paragraph, sentence,

1:08:08word matters because that is a thing that is like, like that is the substance. It's not like some separate thing that is produced out of the writing. And so it any, it's a, it can't just be, it can't like be slop in the way that like code can be slop and still produce some outcome that you want.

1:08:25But you were just pointing out how actually we've gotten much better at agents writing not just functional code but clean code. Why is it not the case that the same progress that allows you to go from merely functional to like clean and like a mergeable PR doesn't also result in like clearer writing?

1:08:41Yeah, that's a good point. I mean, also... Has it not? Like I agree there's many ways in which they're terrible writers, but for a lot of writing I consume, I find it's better to just copy paste it into an LLM and just say like, explain this to me.

1:08:56The explanation will be better than the thing that is produced uh by the human so it's funny that we say like these are such terrible writers and also my reveal preference is just like can i just have another one explain it even when i'm talking to

1:09:07a human expert like live on a call um if it's a piece of knowledge they have that only they have that's not encoded in uh the distribution i want them to explain it to me but then if in order to understand that i need to understand a more basic concept

1:09:20I would prefer if it was socially acceptable for me to just be able to say, let's pause here, I'm just going to ask another alum how that works and then we can come back to your special piece of knowledge.

1:09:32It sounds, I mean, that's distillation, right, and explanation. And so if you're, if I'm thinking of like quality of you as an essay writer, if it's that I give you a book to read and I want a book report, right, then I might believe that, okay, the LLM maybe gives me a better book report.

1:09:48But I think what we, what people are really getting at when they say it's bad at writing, like what is writing? It's not just distillation of pre-existing ideas. It's not just like how do you explain clearly because they are good explainers. It's like, What is the insight?

1:10:01And this is where it gets like just auto regression is a very weird way to generate stuff because like when you're writing you sort of you sort of know in order for it to be good you have to have an element of the unpredictable and it's not just like increasing

1:10:15It's like knowing exactly the correct point when you want to make an unpredictable move and that that's going to be what's more insightful. And so even if it's like better at explaining a pre-existing thing, it's like what generated that book that you wanted distilled in the first place, right?

1:10:29it wasn't an LLM that like generated it and you just needed it it's like some author who threw a lot of exploration of ideas in the world and then deciding what aspects of it were interesting and which ways of presenting it were like the coherent well motivated narrative it's like they put that all together

1:10:46in some way and you know if they're a good author it's probably one that actually you would err on the side of reading their book instead of the distillation but still what makes it worthwhile to like Explore at all in the first place and you're uploading it at all I think it's all of

1:10:59that side of it that's the like when people will cite them being bad at writing and it's that element of unpredictability of being deliberately choosing something that's novel that's like very directly contradictory to like the way that things are being produced.

1:11:12Yeah, that's a good point. I think they're also really bad at building really good mental models of people, which I think is a very important skill in writing. So Annie Matuszak and another collaborator, whose name I'm forgetting right now, did an interesting report where they tried to teach LLMs to write good space repetition prompts.

1:11:29And I really like this because even though it seems like a really totally random skill, it's just like people are talking about recursive self-improvement in a year. Yeah. And you can't get these things to write good flashcards. And what's going on there, right? Right. And they tried many different kinds of techniques and they're like,

1:11:46you know, sophisticated people. Like they tried to RL open source models. They tried all kinds of including chain of thought and the big prompt they sent to the best closed source model, etc. And the key constraint, it seemed to me, was that writing a good card is about

1:12:03Projecting somebody's mind in three months and what is the way in which they will associate the question, like what kind of answer we'll be thinking about at the moment and is the elicitation that inspires the detail you actually want to take away from the passage you're trying to make cards about?

1:12:22I think writing also is similar to this where if you're writing something you're like the reason it's such a enervating process that takes so long is each word you should be thinking or each sentence you should be thinking what is happening in my reader's mind right now even if I flip the phrasing around

1:12:37So the end phrase goes to the beginning and like this is the first image that comes to your mind before you read the rest of the sentence. That kind of maybe auto-aggression is bad at that kind of maybe a more diffusion-like property of considering the whole rather than going sentence by sentence.

1:12:51But also I think that requires a lot of mentalizing, which these models weirdly struggle at.

1:12:55Well, I mean, interesting question. Like, is it weird that they struggle at that? So... I might butcher this. You know how when you cite studies that you once read and it's like maybe the study wasn't real or something? This is one very memorable one on... Okay, so let's say you want to quiz people's EQ.

1:13:14Like you show a flashcard of someone's facial expression and someone's trying to describe what's that emotion. So I put these really good tests online that'll have a face and then four possible emotions. It's like surprisingly hard to like describe exactly the correct emotion

1:13:30but you also get the sense there really is a correct answer and if you try this with like people in your life you'll notice that the ones who actually are pretty plugged in socially like do really well on it and the ones

1:13:38who are a little bit more like left brain like don't okay so that is a kind of test you can do I vaguely remember an experiment to this effect where they took people who had freshly gotten like Botox in some way and they did like a pre-test and a post-test and like post-test they were just much worse

1:13:54at like reading people's expressions like that feels kind of weird they got Botox and taking the test. It's like, so you do the test and then you go and you get Botox and your face is all like frozen. And now you are worse at understanding the emotions of what you see, right?

1:14:08And the thought is that part of understanding like this emotion that you're looking at is doing it yourself. Like at a facial level, like you're moving your face muscles and it's like you see that, you mimic that and you're like, oh yeah, that's anxiety, right? At some like very subconscious level. So in that sense,

1:14:27If it is the case that models have bad theory of mind, sure, they know everything because they've read what everyone wrote. But at a level of actually able to put themselves in your shoes, in the same way that my face muscles are mimicking your face muscles, that's what helps me understand how you feel. Not surprising at all.

1:14:43They don't have face muscles. Their brain works completely different. It's like an alien trying to empathize. How could it have theory of mind? It would be like this very emergent thing to have theory of mind. Whereas we can just plug it into our own minds. It's like we've got the ready-made hardware to just place it in.

1:14:58That's very interesting. From that lens, it's not that surprising.

1:15:03Okay, Grant, we are both partners with Jamestreet. I'm sure over the years you've interacted with a lot of Jamestreeters. What have you found that's unique about them or their culture?

1:15:11I did this interview with them this year that partly was interesting because they don't usually have anything outward facing I mean in the industry they're known as having like a pretty wild retention rate like people just stay there and I think getting an inside view of that I remember one of the comments someone was saying even though

1:15:26People have role titles like researcher or trader or engineer. They often don't know what their colleagues' actual role is because everyone's doing a little bit of everything else. Even if you're officially a trader, you're doing a lot of research. Even if you're officially a researcher, you're doing a lot of coding.

1:15:40And I suspect maybe that's part of why they have the insane retention that they do because anyone who wants to be growing, they just have the chance to do a lot of different kinds of things.

1:15:49Alright Grant, I'll do the plug for you this time. If you want to watch this full sit-down interview that Grant did with some of the folks there, go to 3b1b.co slash Janestreet. Alright Grant, let's talk more about AI and math.

1:16:02What advice do you have about using LLMs to learn?

1:16:07As I was describing, for a lot of well-known concepts, I find them very helpful, but often just a couple of further messages down and I'm trying to understand something, and they're so confused themselves, they're confusing me, and they don't explain it the right way,

1:16:24and then I know that talking to the right human could clear up my confusion in three minutes. I feel like more and more we're going to want to use these things as somebody who's taught a lot about education and representation and stuff. We're going to want to use these things to learn things.

1:16:42Have you noticed the ways to use them more productively to understand concepts?

1:16:45I'm curious to hear your take on this. I'll give mine. Even pre-LLM, I feel like a relevant insight in learning was recognizing that who matters more than what. So advice to any college student when they're choosing what courses to take is Care a little bit less about your pre-existing interests because they're kind

1:17:03of arbitrary right now and care a little bit more about whether like the person teaching it is a good educator and someone you resonate with I think in choosing what to read like what books to read like who the author is maybe matters more than if it's a prior interest so if there's

1:17:16a book you've liked before read what else that author has written rather than reading another thing on that subject And I'm getting to like LLMs on this. So like there's a difference in feel for trying to learn something if you look at

1:17:28a Wikipedia page of it versus if you look at let's say like it's a philosophy topic and you go to the Stanford Encyclopedia of Philosophy or if it's a math topic you go to the like Princeton Compendium of Math where the difference there is like the articles are deliberately written by one individual who like

1:17:46Tries to actually craft a motivation around it and everything. Whereas Wikipedia, it's this like local minimum that's reached where basically every sentence has to be correct. And I think a good exposition, you care a little bit less about like correctness on the way, but you can like deliberately craft things that are a little bit wrong

1:18:03that you correct along the way that gets like edited out in a crowdsource environment. So like that. LLM Explanations feel to me at the moment a lot like Wikipedia, which is to say amazing, right? Like imagine world before Wikipedia, like how long it would take to like find and like susan and everything.

1:18:20But nevertheless, what's the most useful part of a Wikipedia page? It's often just the references at the bottom, right? You look at the like key references and you go to them and you read them. It's like actually sometimes that gives a much like better overview of it. So

1:18:33Often I like to just ask an LLM, like, who should I read? And maybe I can even give some specifics on ways I want to learn. I actually got gaslit by this once where I remember trying to learn about like semiconductors or something. I was like, this feels very visual. This is all like text.

1:18:48I'm like, is there any really good like well-visualized math video? Or not math, sorry. A well-visualized video kind of like explaining the concepts that you're getting at. And Claude was like, yeah, here's a couple in the top one. It was like, here's one from Three Blue and Brown. I'm like, I can guarantee that there's not.

1:19:06And it was an actual video, an actual link, but it just had like misattributed someone else's. And it was good. And it was like I had a much better experience clicking over and watching that video to learn about the thing rather than like trying to proceed forward with questions there. So in that sense,

1:19:20basically using it like a very souped up version of Google on like zero in on the right human written resource. What about you? Like you engage with these a lot.

1:19:29What's the best way to learn from that? I think you put your finger on it. The most productive learning sessions I've had is when there's Some artifact that a human has produced, whether it's an article, a book, a video, that organizes the relevant concepts in the correct way and builds up the motivation

1:19:44of why building up the next idea would be relevant to solving the next problem you did encounter and the next idea and the next idea. And then using the LLMs to just do a little bit pruning around this branch that the book has identified. So I was going through...

1:20:04I think you might have recommended Steven Strogatz's textbook on... The Chaos one? Yeah. Chaos on Nonlinear Dynamics. Yeah. I love that book. And so I was going through it. And... It was like bliss. It was like your videos in like a book form. He's so good. It was super fun.

1:20:20And the way I was learning it is like I'd have on one third of the screen his like lecture from university. On one third of the screen I'd have that part of the textbook and on one third of the screen I have an LLM.

1:20:30And I was actually thinking if I was back in college and watching this lecture live, it would just totally go over my head. Like these kids must be really smart. because I'm like pausing and like reading the textbook and talking about LLMs and then restarting again but with him curating what is the right order

1:20:46to understand concepts what is the right problem to motivate understanding a concept oh also another thing LLMs are really bad at is a thing a really good human can do is when you ask a question they say like actually you're just like not really thinking

1:21:00about this topic the correct way like the question you want to be asking the correct way to organize these concepts is X and LLMs just can't really do that

1:21:08Yeah, it's a little too placated. I mean, this is ultimately like the very, like the supplicants, you know, that's very like, oh, what an insightful question, you know, that kind of thing. You want to strip that down. That's a good point. And I think that cuts to theory of mind a little bit.

1:21:24Like recognizing that to ask a certain kind of question, Reveals that the mental structures are not, at least they're not the same as what the like explainer has. And sometimes people do this to a fault, right? Like I think a really good teacher, let's say you have like a middle school like math classroom or something.

1:21:41If a student like asks a question that suggests they're thinking about it in a different way. It's actually really hard to take seriously in the moment, hang on, could you get to a right answer with that before you say, oh, instead of that, let's do this.

1:21:54And the really good teachers are able to jujitsu the creative way that the student was thinking about it and bring it in. I mean, LLMs aren't doing that when they are not reframing your question. Instead, they kind of run off.

1:22:11right but the very least it feels like there's three levels here and so like lom is at one good explainers that another but then like the a plus explainer is the one who can like jujitsu your way of thinking um and say like oh that's that's where that's useful

1:22:23and so maybe there is a certain you know cycle all the way around where again five years from now the llms will still be doing that but in the better way what is your recommendation to

1:22:35to students who I'm sure email.

1:23:06I wouldn't trust any advice that I give. It would maybe be how I'd, like, couch it. Even pre-AI, it feels very important for any job that you're going to go into to really understand... Like, if we're talking about a job, right? We're not talking about, like, you're a gentleman scientist and you want to,

1:23:22like, engage with the math world or something. You should understand where the money's coming from and, like, what value you're actually adding and, like, the connection between those two. And I think often, like, a surprisingly small amount of thought is put towards that. Especially students, they're in this environment where they...

1:23:37They probably want to go into math because they've always been good at it and they've just been rewarded in life for like proceeding through the next hoop correctly and next step and when they think they want to be a mathematician it's because it's a version of getting to continue to engage with

1:23:48that it's like well I'll go like where do people get to do this rather than thinking like What value am I adding to other people? And to what extent is that the reason that salary is flowing in my direction? It's actually quite different in different cases. In some cases, it's a very prestigious mathematician,

1:24:05and their presence at a university lends a certain brand value. That's like why the university like wants them. In some cases it's like the NSF grant is given because you've got this like public good belief that we have that basic science has and like you've got this institution around that

1:24:20and there's gonna be this whole bureaucracy around trying to as a proxy for what we think that public good is and a whole song -and -dance around how to like correctly um make them predict that your progress will be in the spirit of that funding sometimes

1:24:33it's just straight up teaching right it's like people like to send their kids to an institute that has experts teaching them and like that's what you're doing and you are providing the brand value by being an expert and then the direct value by like being a teacher so regardless

1:24:47list of whether ai's are like proving theorems or not or like whether we're talking in 2016 or 2026 like that is a thing that not enough students thinking I want to be a mathematician think about but I think it's worth thinking about like for me I think

1:24:59that you know it's I just like wasn't necessarily thinking about it and kind of stumbled into this career path where basically math exploration can be monetized as entertainment right and I like stumbled into that I'm like very grateful that I did but it was an accident it wasn't like this deliberate thing

1:25:14and I think I could have avoided relying on serendipity and maybe done that a little bit more by design and had I been like thinking critically about it so to your question if it's the case that you have almost automated theorem proving and then let's say it's the case they're also really good explainers

1:25:31so it's like even to get the human understanding I think a lot of the like social role that mathematicians serve actually doesn't change that much right you still have a sense of as a public we sort of feel like there's value to basic science and we're trusting in the judgment of mathematicians to determine like where they're

1:25:49is best spent and the prestige comes from within that community it's like other members saying that this was a really good result more than it is like the grant writer who like really understands algebraic number theory to understand that it's a good result and so there's going to be some inner culture of what constitutes like

1:26:31I actually think teaching is one of the most stable post-AGI jobs that there is because it's so relational. This is where parents want to spend their money if they have an abundance of wealth is on good teaching and good educating. And it goes so far beyond explanations. Like even if LLMs are good explainers,

1:26:49the thing that a teacher is doing is such a social like coaching mentor type thing that like that's probably the most one of the most stable careers that's going to exist over the next 50 years. And so insofar as what a lot of mathematicians role is like overlaps with that.

1:27:05You know, you as the prospective student going into it, you could lean into that. Actually think a lot more students should like think about and give like pay credence to the idea of being like just a math educator and like the value that that can serve towards the next generation. So I'll couch again on,

1:27:21I don't think I'm the one to say, here, prospective young mathematician, here's how you should think about the future, because I'm like a YouTuber, right? I'm someone who is not in the institution that they are thinking of going into, and so I'm speaking as an outsider looking in. But it feels like, generally,

1:27:38Good universal advice Know where the money is coming from Know where you plug into that And like if you're just asking those questions You're actually already like steps ahead of all of the other Like fledgling prospective mathematicians Yeah and

1:27:51in fact I think In the crazy world, in the world where within five, 10 years, the AIs are coming up with not only solutions to the Millennium Prize problems, but coming up with just totally novel problems to be solving in the first place, novel mathematical fields and objects and stuff.

1:28:09It is in that world where first of all there's a ton of abundance and two the things that AI minds will have like gone furthest in where they will have seen like furthest beyond our horizons will be mathematics and there will be so much demand of like what have the AI seen can you explain it

1:28:28to us yeah yeah I feel like in the in that world if there's any jobs whatsoever surely distilling what the AIs have learned will be one of them also it's funny because

1:28:39All of this sort of presumes that it's useless, right? Like we're not talking about the actual practical applications of what math is being done. So insofar as there's any economic utility to it, you would imagine that the people who understand it and are able to like make

1:28:52the decision of where it should point, like they actually have a lot a lot more economic value by being able to make that judgment as curator and point this behemoth of new math pointed in a useful direction. Suddenly, that's a much more levered move to make than it had been previously.

1:29:06Can I actually ask you about that? Obviously, one question for AI for math is not only can it do it, but is it any good? or isn't any good for anything. You were describing all the ways in which group theory, we're trying to solve this, we're trying to figure out random facts about the roots of different kinds

1:29:27of functions and now it's all these different applications that are practical across many different fields. Do you have some sense of if we just totally get to a place where mathematics is, the field of human mathematics is accelerated 10x or 100x that some crazy shit happens or are we just actually going to be bottlenecked by

1:29:48I think there's some fields that probably will, I mean, it's super spiky, right? I think like progress in algebraic number theory, it feels unlikely that that then like unlocks some thing. But I don't know, I remember talking to this mathematician who does more like dynamics and like PDE solving

1:30:10and he was referencing basically like his group had some ideas that let me see if I summarize this right it's like the way that Boeing would make planes is they would like make it and then they would do a bunch of tests and they had to like disassemble it and reassemble it based

1:30:25on those tests and they essentially had some insights on how to like do more things in simulation such that you don't have to like deconstruct and rebuild it and it saved Boeing just like billions of dollars or something and then they just started funding that like group which is so that's it's like

1:30:39Much more obviously application adjacent because like PDEs just sort of are that. So progress in that domain. You would imagine like actually do unlock some things and I don't know if it's these like step changes but maybe it's more on the side of like engine design becomes just a little bit more fluid

1:30:57or you know like coming up with the right wing shape instead of running a whole bunch of complicated like CFD or maybe you're able to like speed up your like CFD simulations because of certain pure math insights like makes those more efficient. I bet you'd just see like a lot of like great incremental improvement there.

1:31:14It seems less likely that the massive breakthroughs in math immediately turn into this massive economic breakthrough. You solve the Navier-Stokes problems and then that unlocks an ability to simulate more things. But you probably will see at those fringes just some Some meaningful leakage outside of the pure math insights into other things. Also, I mean,

1:31:39there's a ton of people working on things like AI engineers, like physical engineers, like material science and things like that, that would be, you have to imagine that they would be in a good position to look at the AI math insights and decide if they're relevant in some way or not. And so...

1:31:57It's another one of these things where I'm not going to sit here and like put a flag in the sand like predicting that there will be. It would be a little bit disappointing and a little bit surprising if there weren't over the next five years like economically valuable improvements that were made

1:32:10that were directly like referable to the like AI progress in math. Like that just would be kind of disappointing if it was just taking down a bunch of Airtish problems and like none of them actually, you know, it wasn't doing any of the math that actually directly touches physical world.

1:32:24Yeah. I mean, to your point about, well, a lot of history and mathematics is about like building up these like piles of concepts and connections and whatever. Yeah. And sometimes the piles connect with each other or you discover an application somewhere else. At the very least, you just build up this huge pile. And as a, you know,

1:32:41broader progress in society happens during the singularity, when like we get to the industrial part of the singularity, you just have all these different ideas that you can hopefully are useful in other parts of the world.

1:32:53I mean, yeah, like I said, one of the interesting things about what's happening is it causes people to step back and ask like, what is math? And maybe one of the awkward conclusions of it will be revealing like, oh man, over the last, like it's just become wholly useless.

1:33:06Like the kind of questions being asked have become like so divorced from. that are physically applicable. That's one of the things mathematicians have to come to terms with, where everyone will look and be like, hang on a second, like, where are you guys supposed to, like,

1:33:18if there's so much, that's like 10x progress there, like, why aren't we seeing it over here?

1:33:22And then MathChurch is like, ugh. Every time we wrote those Grant proposals and said, like, trust us, like, the elliptic curve progress is going to help with, like, cryptography. Like, it, like, shines a light on the fact that, like, maybe it doesn't. So that's one possibility. Grant, this is super fun. Thanks so much for doing it. Absolutely.

1:33:38My pleasure.

Always so much fun to chat with Grant. AI has been making much faster progress in math than in other fields. As a result, mathematics is showing us, very concretely, what AI progress in other fields will look like. Even within mathematics, there’s a jagged landscape. What does it look like? What is the nature of the most important conceptual breakthroughs in the history of mathematics, and how different are they from what AIs are currently able to do? Does AI (on net) increase or decrease human understanding of the field? How big is the overhang from having AIs systematically try to connect ideas already in the literature? And what advice does Grant have for aspiring mathematicians, coders, and other students who are passionate about fields that are being most transformed upon by AI? Watch on YouTube; listen on Apple Podcasts or Spotify.

(00:00:00) – AI is discovering new proofs. Is that AGI? (00:11:32) – The verification loop on conceptual breakthroughs can be a century long (00:26:12) – Will we understand an AI proof of the Riemann hypothesis? (00:38:08) – Can AI find the hidden bridges between fields? (00:53:48) – Why real-world tasks don’t fit into RL environments (01:07:07) – Good writing requires theory of mind that AI still lacks (01:16:02) – Why learning will still depend on human curation Transcript

00:00:00 – AI is discovering new proofs. Is that AGI?

Dwarkesh Patel Today, I’m chatting with Grant Sanderson, who runs 3Blue1Brown and is now working on a new project documenting the progress AI is making in math. I wanted to talk to you about this because AI has been making the fastest progress in mathematics out of any other field. Whatever is happening here, and whatever way we’re seeing AI progress happen or not happen, will tell us about what will happen to the rest of the world as AI gets better and better. I wanted to start with this question I asked you when I first interviewed you three years ago. I asked you, once we have AIs that can get gold in the International Math Olympiad, wouldn’t that just be AGI? Wouldn’t this just be able to do anything any human can do, given how hard these problems are? You had an answer, which in retrospect turned out to be very wise and correct. You said it’ll be another benchmark, like all these other benchmarks that AI are passing. Obviously, AI has gotten better in a general way since then, but there won’t be some “aha” moment when this happens. First, I’d be curious to get your heuristics on why that turned out to be true. Second, I’m curious how long you think this narrowness can continue to be true. By the point that AI has solved a Millennium Prize problem, do you think it’s still possible that there are lots of tasks humans are doing that AI still can’t automate in the economy? Grant Sanderson It’s an interesting question because it’s hard to answer without knowing what the solution looks like ahead of time. If we take the IMO, the spirit of your question three years ago was in looking at how some of the solutions to these problems really seem to require creativity. The designers of these problems try to come up with things that you can’t train for as easily. The dirty secret with the IMO is that you really can train for a lot of them. With the whole AI and math project underway, as you point out, one of the reasons it’s interesting at all is that there’s a spiky frontier to AI, and math is just right there in one of the spikes. But there’s a fractal nature to that spikiness, because when you zoom into the specific progress within math, you have some things that are a lot easier than others. If we just think about IMO, which is old news at this point. It’s been two years since they’re really doing quite well. They would have gotten a gold in 2024 if not for the following reason. They’re very good. They just cold-solved geometry basically. The IMO has these four categories of problems: geometry, number theory, algebra, and combinatorics. Geometry, it just solves it in nineteen seconds since 2024 because it’s a brute force solver. The dirty secret is that for students, there’s also a brute force way you can go at it. Combinatorics is the wild card: much more playful, puzzly-seeming problems. There were two combinatorics problems on that year’s test, and there’s not always. There are four categories and six different problems, so it’s a toss-up which one is going to have two questions. Had it been more geometry questions, they would have gotten a gold that year. But it struggles on those combinatorics ones. Someone who’s trying to keep that torch of the last holdout of math for humanity might say those are the ones that require more creativity. Even then, the spirit of your question—if they’re solving a Millennium Prize problem, does that also service a lot of white-collar work?—suggests that whatever the rate limiter is between where we are now and that is the same as the rate limiter for making things better at white-collar work. We could paint a couple of different ways. If we focus on the Riemann hypothesis, what would it look like to solve that? These things are extremely good at a specific domain of knowledge, knowing it very deeply, and then knowing another domain, and another. You’ve pointed this out. It’s bizarre to have something with this superhuman breadth that knows all the fields so well, and yet isn’t finding those lightning bolts that connect them. I think we’re starting to see sparks of it actually finding connections between the things it’s an expert at. I’m sure we’ll talk about it. If the nature of the solution to the Riemann hypothesis was something like that, that feels pretty distinct to me from what’s necessary to get good at white-collar work. And there’s a reason to believe that might be the nature of the solution. I don’t know if you know the story of Hugh Montgomery and Freeman Dyson at the IAS. This is a side tangent, but it’s a fun story. I don’t know if it was over lunch or something like that, but you have this number theorist who is just trying to understand the statistical correlation between pairs of zeros of the Riemann zeta function. The Riemann hypothesis is all about whether all these zeros sit on a straight line. He finds this quantitative question you could ask, and he writes down a formula. It looks like one over sine squared or something like that. Freeman Dyson, a physicist, is like, “I know that expression. That expression comes up in studying the eigenvalues for random Hermitian matrices,” which was something that comes up in studying the energy levels of a nucleus. The idea that the statistics of those two seemingly different things were the same prompted an exploration of whether there are aspects of random matrix theory that might be relevant to the Riemann zeta function. I think it’s a little bit of an open question whether there is fruit to be had there. But that bridging together of two different fields—if it turned out that the solution to the Riemann hypothesis was exploring an idea like that even further—has the character of how you expect LLMs to be good at math. They’re experts at quantum physics. They’re experts at analytic number theory. They should be able to see that similarity in a way that doesn’t require Montgomery and Dyson to be having lunch and happen to talk about it. That’s totally different from white-collar work. To the extent that you have a hard time using an AI as an editor, it’s not because they know everything and you just need them to find that lightning bolt in between. A different possibility would be… What’s the right analogy? Maybe if we think of Fermat’s Last Theorem, between the moment of Fermat phrasing the question and what the solution itself looks like, where the solution ultimately involves such heavy machinery in math. The beauty of that problem is you can phrase it so simply. You ask about xn + yn = zn. Do you have integer solutions for this when n is bigger than three? It’s something you might expect there to be an elementary number theory approach to, but as far as we can tell, there’s just not. Whereas the actual solution, maybe there is something simpler, but this might be what it has to be. There’s such a complicated set of ideas that build on centuries of work centered around elliptic curves. Then there’s this other mountain of ideas centered around these things called modular forms. Both of those mountains have to be built before you can ask the right question that connects them. If the solution to the Riemann hypothesis involved building a new mountain, that’s a kind of skill—the ability to come up with the right new ideas—that feels sufficiently different from the character of how they’re intelligent right now. It’s not like that’s what you need from your hired video editor per se. But if it’s capable of building mountains that are the correct new theory crystallizing how we should be thinking about a subject, that’s just such a level of intelligence that it would be surprising if it didn’t permeate into other aspects of the economy besides just the mountain-building for math itself. Dwarkesh Patel Or at the very least, even if it couldn’t literally do every single thing white-collar humans can do, it would just have transformative effects in the way that getting gold in the IMO did not have transformative effects on the world. First of all, I do want to point out that I’m totally moving the goalpost here. When I interviewed Dario two or three years ago, I asked this question about why they haven’t been able to use their vast knowledge to connect ideas together and come up with a new discovery that way. That seems like the kind of thing where even a moderately intelligent person, if they knew this much information, would be able to come up with a medical diagnosis from the fact that this drug causes migraines, and this other thing does this, and maybe it’s the same drug that can cure both things. From an outsider’s perspective, mathematics seems clearly like a field where finding the counterexample to the unit distance problem conjecture was an example of this kind of thing. So it’s total goalpost moving. But then we can ask, what is the next benchmark? Now that AIs can do this thing we should have thought they’d be able to do, what is the next thing that would be quite impressive? There are a couple of candidate ideas here. One could be coming up with interesting problems in the first place, and the other is coming up with new kinds of objects or conceptualizations that create or unify fields. On the first one, right now we have these Millennium Prize problems because mathematicians have noted them. Riemann came up with this idea of the Riemann zeta function because he thought the zeros of this function would have some connection to the density of prime numbers. Figuring out why we think this is an interesting thing to study in the first place, why we are building this object and trying to answer questions about it—and answer this particular question about it—seems like the kind of thing that would be the next benchmark. Grant Sanderson You highlight two pretty good examples there. For anyone curious about the unit distance conjecture, there’s this really nice video by a math channel called Polylog where they talk about it. All of these discussions cause people to reflect on the process of doing math. They’re like, “Oh, this thing can do this impressive stuff. What does that mean for us?” One of the people in that video highlights this quote: “good mathematicians prove theorems, great mathematicians come up with conjectures, and the greatest mathematicians come up with definitions.” That’s more or less exactly your framing here. We need the conjecture generator and then the definition generator. That’s the premium-tier mathematician. I don’t understand how exactly you’d make that a benchmark. Usually, when I think of the word benchmark, I’m thinking of something that is a goalpost. The ball is through the goal or it’s not. You can clearly say, “Yes, this is done.” Partly that’s to be able to do things like RLVR, but also partly just to know that you haven’t moved the goalpost in answering. OpenAI can have their headline on disproving the unit distance conjecture because it’s a clear, distinct thing. It did it. Whereas imagine trying to have a headline on GPT-5.4 coming up with a really good conjecture. “We promise, everyone thinks it’s a good conjecture.” It just doesn’t land the same way. But maybe that doesn’t negate the fact that it’s the right thing to be thinking about. I would be surprised if it ever took the form of looking like a benchmark, where we have a score saying it’s passed because we can quantify how good a conjecture is. The nature of what it would take is probably that you’d feel a tone shift in conversations with mathematicians about the way it’s useful to work with. This series you referenced, which is not at all produced yet and probably won’t be for a couple of months, takes the form of us interviewing a lot of mathematicians. What’s interesting is that we started doing this over a year ago, and it’s fun to see a little bit of a tone shift in the way they talk about AI between mid-2025 and where we are now in 2026. In the real world, that’s a very short amount of time. In the AI world, that’s eons. We’re able to see this tone shift over those eons. I think the way you’d measure conjecture-generating ability is going to be more subjective, based on that tone shift. It will be mathematicians saying they’re not just using it to solve their problems, but that as they step back and decide what their research field should even be, a conversation with such-and-such model was genuinely helpful for that. I don’t think it’s likely you’d see it in the form of a headline saying this was yet another benchmark knocked down. 00:11:32 – The verification loop on conceptual breakthroughs can be a century long

Dwarkesh Patel It’s very interesting. The kinds of things you can’t make benchmarks for are also the kinds of things, at least in the current paradigm, you can’t easily train for. There’s really no fundamental difference between a benchmark and a training environment. It’s very easy to come up with some dichotomy of, “here’s a deep reason why AI can’t do a certain thing”, and then it turns out you’re just thinking about it the wrong way, and actually it can do it pretty soon thereafter. But I’m going to come up with— Grant Sanderson You’re going to come up with a couple anyway. Dwarkesh Patel It’ll probably turn out that there are ways we can train AIs to do these kinds of things in the relatively near term. But it seems like it would have to be different from current RLVR training. The thing I’m curious about—and the thing that seems to me to drive a lot of the big progress in mathematics and in science generally—is coming up with a new way to think about a problem or a new way to understand the world that unifies different fields, spawns entire new fields, and solves problems we weren’t even trying to solve in the first place. The reason Einstein was thinking about GR is not because he wanted to explain why light bends or why black holes exist. These are phenomena he didn’t even need explained in the first place. In mathematics, as a total outsider who doesn’t even know what he’s talking about here, it seems like there are often ways to prove a specific problem that can motivate a new conceptualization—one which results in a whole new field, a whole new way of thinking, which is immensely productive—and ways which don’t. I’d be curious to hear you talk about Galois coming up with group theory, distinguishing his solution to the quintic having no formula for the roots, and Abel coming up with a different proof a few years earlier that didn’t come up with group theory. If you wanted to do a verification loop on whether group theory is an interesting concept—was something useful done here, or why is this proof better?—potentially that verification loop is a hundred years long. It involves cryptography coming around and physics making progress, and the ideas in group theory being relevant to understanding symmetries in physics. There’s a hundred-year verification loop on why this is a productive concept in the first place. Grant Sanderson You struck a nerve, because I had this project about Galois I was going to do in 2022 that I put on the shelf, but I spent a year of my life thinking a lot about what he did. There’s a risk of me accidentally talking too long on the specifics, which you can hold me back on. It’s a perfect example for your case, because describing why it was a valuable insight does not come from immediate utility. Certainly, if you’re thinking about RLVR environments, this is going to be really hard to do. But it’s interesting to note that even with human verifiers at the time, it took a really long time to recognize it as being useful. With Einstein and GR, people could feel this was a good theory right away. What makes Galois theory such an interesting example is that you literally have this hundred-year segment of an idea that flows through many different people’s heads before it settles into something the math community agrees is good. To back up a little bit… Do you want the background on the problem at all? We all learn about the quadratic formula in school. Dwarkesh Patel I thought you were going to say we all learn about group theory in school, but I missed that class. Grant Sanderson We all learn about group theory… No, the quadratic formula. This was known. In some sense, the Greeks could solve quadratics, but they didn’t really write things in algebra. It’s really the Arabs who wrote down that formula. There’s this delightful story about dueling Italian mathematicians—not real duels, just intellectual challenges—who secretly found a formula for the cubic, and then very shortly thereafter found a formula for degree-four polynomials. So a natural open question for mathematicians is, can you find a formula that solves degree-five equations? The degree-four formula is a monster. It would be wild to write it down. You usually don’t write it down in full. You break it up as a procedural thing. You might believe these things have this exponentially increasing complexity. So for many hundreds of years, nobody was really answering that question. Usually, we say Abel was the first to prove it. He was this young, precocious Norwegian mathematician. He showed it’s simply impossible. It’s not that you can find a quintic formula. He thought he found one initially, but he showed it’s impossible. I think the real credit though, you have to back up a bit and talk about Lagrange. He found the right kind of question to ask about this. I’ll give it at a very high level. He was studying the question and recognized that being able to solve these polynomials is very related to understanding the way certain algebraic expressions are symmetric. If I write down a + b + c + d, just adding four variables, and I permute those, it doesn’t change the value of the expression. Whereas if I write a + b * c + d, some of the permutations don’t change it, but some of them do. He had this really nice insight about how if you can find expressions that have four free variables, but all the permutations take on three distinct values, that has this unexpected relationship with being able to reduce degree four into degree three. He started approaching the question of whether we can find a quintic polynomial by wondering if he could extend that method. To extend that method, you would have to have an expression that has five free variables such that as you permute them over all the five factorial permutations, it takes on only four values or fewer. You could put that in a puzzle book. You could put that in a brain teaser that a twelve-year-old could engage with. It’s not too hard to find yourself feeling like that’s an impossible task. Lagrange is sitting there saying, “Here is a strategy to solve this problem of finding a quintic polynomial. It seems like it might be impossible, at least from this strategy.” But that was the first time in history that people had the instinct that some kind of question about symmetry was the right way to study these polynomials. In his mind, it was just a way. It had yet to be discovered that there was actually a tighter connection. Also maybe rather than searching for the formula, we should be asking the opposite question: can you prove that it’s impossible? He sort of planted that seed. Around fifty years later, Abel definitely read Lagrange and was influenced by it. We know that Galois loved Lagrange when he was falling in love with math. It’s very hard to imagine that these two young geniuses coming up with pretty similar insights around that problem wasn’t born from Lagrange. But to your question on whether you are able to verify that this was a good idea, there wasn’t any result that Lagrange came to. He didn’t solve the problem, so it wasn’t a case of knowing it was the right question to ask based on a solution. He just asked it. There’s some intrinsically interesting thing about it. It also wasn’t very important for math at the time. Most people were more interested in the applications to physics. This was almost a side, recreational, hobbyist-type thing. Abel started working on quintic stuff, but then he was advised to spend more of his efforts studying elliptic functions, so more of his work was on that before he died young. He died at twenty-six from tuberculosis. And then Galois pushed both of those ideas in the right direction, where he really understood the nature of abstraction. He had this really nice piece that he wrote while he was in prison. We could talk all about his life story. It’s pretty wild. But he’s this teenager, he’s in prison, and he had tried to submit his math papers and they had been rejected. So again, thinking about verifiable reward, the verifier function that is the academy at that time is rejecting what he wrote. Frankly, it was not very coherent. It wasn’t a complete proof. He wasn’t giving a clear thought of what the theory actually was. He was just a young fledgling mathematician getting his bearings. The verified reward there is, “No good.” But he has some instinct that there’s something there. So he’s writing this diatribe on the nature of math being something that undergoes these shifts over time. He talks about the advent of algebra itself and going from just thinking in terms of numbers to having a certain fluency with pure algebraic expressions, where you’re not tied to interpreting those expressions. He has this instinct that there seems to be another layer of abstraction that we should be doing, where rather than thinking about the formulas themselves, we’re thinking about what symmetries underlie those formulas. But it was still a pretty ill-defined theory. If you’re trying to say the verified reward is that he solved a problem that other people haven’t, well, Abel already proved that quintics are unsolvable. So what was Galois doing? In principle, Galois theory lets you take a specific polynomial, and it gives you the rules to say whether that specific polynomial has roots that you could write down. For example, with x5 - 1, you know that a solution is 1. Or x5 - 2, you can write down the fifth root of two. So it’s not that you can’t write down the solution for every quintic polynomial, but could you find a specific one where you prove you can’t write the solution using radicals? He also didn’t even solve that exactly. He didn’t show for a specific example that he couldn’t. Even describing what problem he solved is very tricky. He then dies. It’s this very romantic story of him having this duel. There’s a lot of myth around how he supposedly writes up all his ideas the night before the duel, but really, he tried to get them published five times before. Dwarkesh Patel Working on the quintic doesn’t seem to be good for your health. Grant Sanderson It’s very bad. If you’re a young genius, don’t work on the quintic. He asks his brother and his close friend to get his notes to Gauss, to get these notes to the important mathematicians of the day, because he thinks there’s something there. Even then, it didn’t really take. His brother and his friend tried to get them out, but it was another twenty years until Liouville sees these notes, sees that maybe there’s something in them, and tries to clean them up and understand what Galois was getting at. Even then, it was another twenty years or so until Jordan actually puts together something like a modern treatment of group theory that they attributed to Galois. You could easily imagine history turning differently, where these ideas were coming about from other points in math, and Galois could have been forgotten in history if he was a less florid character. But between the time of Lagrange having this inkling that maybe symmetries of roots is the right way to go, to where it all looks like modern group theory, you’ve got this long span. A lot of the time, it’s not even passing the verified reward of human reviewers. It gets on someone’s desk and they say, “I don’t really know if there’s anything here.” You have to have this one person recognize it. Even then, it’s not really solving practical problems at that point. You pointed out cryptography and physics and things like that. You have to get into the twentieth century before you have Gell-Mann thinking that maybe understanding the nature of how certain groups break down has a relationship with what particles are made out of. He anticipates quarks based on a purely group-theoretic question. That’s one of the more interesting applications of group theory: to even predict the existence of quarks is a group-theoretic question. That’s so long after Lagrange before you have anything like that. So you have to ask, what is the way of measuring progress that’s not based on solving a problem, but that is somehow capturing the instinct inside Galois’s mind when he says, “I think there’s something here”? What’s the instinct inside Lagrange’s mind when he says, “I think this is the right way to think about it”? What’s the instinct inside Liouville’s mind when he says, “These scattered notes from this long-dead youngster might have something to them”? It’s so hard to put a finger on that. A different series of videos I’m making right now is about the whole “compression is intelligence” idea. Even though this isn’t really the angle I’m taking, there is something to the idea that the smaller expression that’s more predictive feels more intelligent. So I wonder the extent to which you can give some kind of verifiable reward around not just whether you solved it or what it is solving, but around the smallness of the concepts required to do it. Going back to Riemann hypothesis solutions, what would that look like if an AI solves it? I think a third way it could happen is it just straight-up works harder. In the same way, you could maybe have an elementary proof of Fermat’s Last Theorem that’s just spelled out over thousands of pages that would be incoherent. But the cleaner way to view it is with elliptic curves and all that. Maybe there’s some thousand-page proof of the Riemann hypothesis that no one’s really getting anything out of, and what you actually want are the succinct, compressed versions of those ideas that would then lend themselves to human understanding. Maybe you throw Kolmogorov complexity into your attempt to quantify what you mean by elegance. I don’t think it’s easy, but I do think it’s something you would have to do in order to reward the Galois-like instinct, rather than just rewarding whether you solved a problem. Dwarkesh Patel It’s very hard to come up with the heuristic for science. But it’s clear humans have been doing this somehow, and obviously, AIs will do it at some point. Grant Sanderson It’s relevant also not just in terms of verified reward, but presumably, the end goal is understanding, human understanding. Even if you do have some thousand-page proof of some math thing or some grand new physical theory, the goal is understanding. Maybe if the goal is predictiveness, you can just have automated engineers go off and build rocket ships where we have no idea how they work, but we can get between stars. But there are going to be a lot of people who want to understand. You’re still going to want whatever the concision function is that distills down this complicated way of thinking into the right one, like the equivalent of the universal law of gravitation for Newton. You would still want to train AIs to be able to do that and find the compressed representation. 00:26:12 – Will we understand an AI proof of the Riemann hypothesis?

Dwarkesh Patel People have this worry about mathematics in particular that AIs will prove the Riemann hypothesis, and our understanding of mathematics won’t be any the better for it. I have a couple of questions about this. The first one is whether this is something you should expect. Isn’t the reason humans come up with general, natural objects and subgoals when we’re working on a big problem that this is just useful when you’re trying to work on a complicated, important problem? Theoretically, would this even be a simpler way to solve the Riemann hypothesis, as opposed to just coming up with the natural abstractions that are relevant to thinking about the problem? And then two, empirically, is this what we observe when AIs make progress on problems today? When the AI came up with that counterexample to the unit distance conjecture, you can just read its chain of thought. It’s not understandable to me, because I don’t know anything about mathematics, but it seems that to other mathematicians it was understandable. It made use of known concepts of mathematics and proved relationships between them, all in natural language. As a result, it accelerated our understanding of the connection between this object and this conjecture. Empirically, is this a thing we should be worried about? Grant Sanderson I think it depends on the nature… If we break down the three possible ways of solving the Riemann hypothesis… The other big one from this year was a certain Erdős problem numbered 1196, about these things called primitive sets. It had that character of bringing an idea from a seemingly different field. As soon as you present the basic idea to a mathematician… You say, “What if we try the Markov chain process where we show that this thing is one from the bottom up probabilistically rather than the top down, and use the von Mangoldt function?” If you say that to someone in the know, they’d know how to run with it. You have this very small idea that has the form of expertise in one field and expertise in another, drawing a little lightning bolt between them. Those are going to be very human-parsable, because all you have to do is show the start and end point of what those connections are. If the character of it is mountain building, you have to put in a lot more time to understand that new mountain that was built, because it’s a new thread, not just a lightning bolt between them. And if the nature of the progress was just raw hustle—a super long chain of reasoning with no new theories—then you would have that worry of this whole digestion process. So I don’t think there’s one clear answer. It depends on what the solution would look like. On the mountain building side, that would actually be really interesting to see. Is it by default very human-understandable, the way we see new theories from great mathematicians? Or is it an alien, different kind of mountain being built where we have to reprocess the kinds of abstractions we engage with? The closest example here would be the attempted solution of the abc conjecture. We maybe shouldn’t get into that one, but it probably is not a correct solution. Basically it’s this whole new way of thinking that this otherwise reputable mathematician in Japan had come up with. It took mathematicians a long time to even parse what he was saying, but it had the feeling of an alien bit of mathematics that’s theory building, not just a long chain of reasoning. He called it inter-universal geometry. The biggest fear would be that an AI does that, and then much like the abc conjecture, people work for years to go up the mountain, and they’re like, “Dang it. This just isn’t right.” If it turns out to be wrong, but it really looked right. Even if it was right, there’s just a lot of effort to hike up a new mountain. Dwarkesh Patel If we end up in that situation, David Bessis had a really great blog post called “The Fall of the Theorem Economy”. He’s talking about how historically, as you were saying, mathematics is coming up with these definitions and problems, and it’s about proving theorems about them. The theorem-proving stuff is what gets all the credit, but it’s really a parasite on the coming-up-with-the-definition stuff. Historically, this has not been a problem in terms of credit apportionment, because if you come up with a definition, you’re probably going to be the guy who comes up with a theorem. But now we’re in a situation where if the valuable work is coming up with the insight and AI automates the latter part… Imagine a scenario where an AI comes up with Abel-like direct arguments about a bunch of important conjectures in the world, and then we just have these proofs. Now it’s up to humans or future AIs to consolidate. Again, having no object-level understanding of this argument whatsoever, I’m sure that if you had access to it, it would make it easier for you to think about what’s going on. Is there some deeper way in which we can understand why this proof works that would make it easier to come up with the ideas behind group theory? Grant Sanderson I think it would be hugely helpful. So much of trying to discover new math is mostly being wrong. You’re trying to solve a problem, and it doesn’t feel like constantly taking the correct step up the mountain. Mostly it feels like a random drunken walk, where you’re doing a thing and then you’re wrong and constantly discovering that. If at the very least you know that trying to digest what you have is ultimately leading to a correct solution, that feels like progress, simply because of the sense of knowing it leads to a solution. There are plenty of instances in the recent history of math where it feels like the reach has exceeded the grasp, where things are proven long before they’re understood. One of my favorite openings to a paper—it’s not even a research paper, it’s more like an expository one—is from a mathematician named Timothy Chow, who was trying to understand a concept called forcing. There’s this problem called the continuum hypothesis that more or less asks: you have a size of infinity for the natural numbers, and a size of infinity for the real numbers. Is there something in between? The answer is both yes and no. It depends on your axioms. It’s outside the scope of our usual axiom systems, which is an interesting answer. But the method to describe it is really hard to understand. It’s this thing called forcing. In the beginning of this paper, he writes, everyone knows the idea of an unsolved research problem. I want to propose the idea of an unsolved expository problem. Sure, we’ve proven it, but we don’t really know why it’s true. Then he proposes a partial solution to that expository problem. You can imagine why I loved that framing, because this is my whole life. I don’t do research math. It’s wholly about what’s the most clear way to understand this, even if it’s proven. There is a difference between proof and explanation, and I think you’re getting at the importance of that distinction. Dwarkesh Patel Yeah. That will be the main incentive. Or the incentive would have to change, not just in mathematics but in other areas of science, from proving things about the world to consolidating proofs into problems or higher-level insights. We were having a discussion earlier at lunch about a recent talk you were giving on design and how it helps us understand things. In the limit, is there really a difference between the conceptualization of an idea and the idea itself? If you think about special relativity and spacetime diagrams, and Minkowski spacetime, this is a way in which we illustrate why there’s length contraction and time dilation. But that is the reality… So, the exposition does seem to be the explanation in some sense here. Grant Sanderson There’s a couple of interesting things there. One is that there seems to be a really strong correlation between the people who come up with genuinely novel insights and the people who are actually quite clear in their communication of it. You might imagine the opposite, given that the experience of a university student is often that the expert teaching them is not necessarily the best explainer of that topic, because they’re so spoiled by their expertise. But what seems, at least in some cases, to be the case is that the people who are really coming up with something quite novel—you’ve got Einstein or Claude Shannon or someone—you read their papers, and they’re really lucid. It doesn’t feel like this is just for the experts and you have to chop through it with a machete. They’re very good expositors. Feynman has this characteristic too, he’s a very good expositor. Maybe the same part of the brain that comes up with the correct new way of thinking about it at a research level also has this knack for good explanation. I think this is pertinent to AI. I used to think that AIs would become these automated theorem provers, but the role of mathematicians was going to shift towards my job, explaining these things. Now I suspect that actually they’ll also be quite good at doing that, probably better than most humans are at explaining and distilling. So digesting and explaining what was going on is probably actually not what’s left for mathematicians, by the nature of how these things are going. We can talk about ways this might not be it, but probably the same thing that comes up with the really good new idea that solves some new problem is also just good at explaining it. That’s a way my beliefs have changed. Dwarkesh Patel What’s the last thing you think you’ll be doing? Both you and also what the human mathematical community will be doing. Grant Sanderson I will probably be doing something like what I am until I die. Dwarkesh Patel If the doomers are right, maybe it’ll be for the same reason. Grant Sanderson Yeah. You build a man a fire, and he’s warm for one night. But set a man on fire, and he’s warm for the rest of his life. So that’s where I am with AI. Some of the function of an explainer or a teacher is to add clarity to a thing that someone’s curious about. That’s one thing. But some of it is more relational, providing motivation and a sense of curation. One interesting take that I’ve heard about what mathematicians will end up being is that it’s actually more analogous to art museum curators than anything else. The AI solved the thing, so the art exists. They even know how to explain it really well. But you still want someone to help you navigate this nearly infinite space of what ideas are worth engaging with. Even if AIs were in some sense better at that, I think we would always still prefer a human that we had a relationship with, because the way we get motivated to be interested in things is a social phenomenon. If you have some specific technology you’re trying to build, that might be different. But the people listening to this podcast trust your curation on what’s an interesting topic in the first place. It’s not that they’re landing here because whatever your next topic is, that’s what they wanted to understand in a prior sense. They’re trusting you as a curator. So my role, and arguably that of other mathematicians, might actually just shift subtly into that curation direction of what ideas are worth pursuing. That’s a lot of my job right now. I think people assume a lot of the time for a video goes into the visuals. Sure, it does. It’s not immediate. But actually a lot of it is just deciding what’s worth saying in the first place, what’s worth putting there. I want to engage with that, and I think I have a trust with certain people, and they’re curious what I would choose to put forward even if the AIs are better than that. It’s the same reason human musicians are always going to have a role: that social function of the story behind them, even if the objective quality of the MP3 file coming out of some model is better. That’s what I see happening to my job. 00:38:08 – Can AI find the hidden bridges between fields?

Dwarkesh Patel I want to go back to a question from earlier. Just as AI has crossed this threshold, this important benchmark of being able to connect existing ideas to come up with a new discovery or prove or disprove something, we’re like, “Okay, but what’s the next thing?” Grant Sanderson There’s a lot more to do on that one, by the way. Just because a couple lightning bolts have been thrown… I think there’s this flourishing future over the next couple years of really connecting. Dwarkesh Patel Right. So in the limit, you could even say—I don’t know if this is accurate, but potentially—a lot of the biggest breakthroughs look like this at some level. With general relativity, you’re just connecting together Riemannian geometry and special relativity. So as AIs keep getting better and better at this connection thing, maybe a lot of big breakthroughs are not really of a different qualitative nature. I don’t know if you have a take on that. Grant Sanderson A lot of the conversation has focused on problem-solving and that nature of math, ticking off Erdős problems or something. But I’d say it’s not even a majority of mathematicians who would characterize their work as really targeting the next problem to tick down. Are you familiar with the Langlands program? Dwarkesh Patel No. Grant Sanderson It’s not even a field of math so much as it is a research ethos. Fermat’s Last Theorem is one inkling of this. You had these two seemingly disparate things, and a connection between them led to a solution. Langlands was a mathematician. He has this famous letter essentially spelling out how it seems likely that there’s a lot more connections like that. He even got a little bit more specific about the nature of the connections, such that you might imagine this large map, and you’ve got this valley over here and this mountain over here and this set of plains over there. There’s a lot of mathematicians who would characterize their work as being part of trying to understand the threads on this map. The progress there, it’s not even “Here’s this one specific problem that we know will be solved by that connection.” It’s more that time and time again, there have been cases where big problems were knocked down by finding connections, such that it’s almost preemptively finding the connections. It’s actually very interesting. Anytime you run into a mathematician, ask them whether the character of their work is more akin to the Langlands program or to targeting one particular problem. You get a certain bifurcated split there. The possibility of AIs being supercharged connectors feels like it might be an amplifying tool in that pursuit. It’s hard to measure, though. This cuts to what we were saying earlier: How do you assign a score to say, “Yes, you’ve done it”? If it’s knocking down a problem, you have a clear way of saying, “Yes, you’ve done it.” You can write the headline. You can have your PR move as the AI company to say, “We did it.” Whereas if it feels like that was the right connection to draw, you can write theorems around it. That’s the nature of what the papers in that field look like. But I think it will require a lot more “human in the loop” to say, “What was the kind of connection that we were going for?” That’s my guess on what most of the useful progress from these models will look like in the next five years. It’s just really filling in that landscape of connections that you can draw if you’re an expert in multiple fields. As you’ve pointed out, it’s kind of surprising we haven’t already had this. I would be curious to know at a technical level what causes the unlock there. On the one hand, you can paint an explanation in your head for why you could be an expert in all of these things and not be drawing those connections. When the method of reasoning is this autoregressive chain-of-thought phenomenon… Autoregression is actually a really weird way to produce stuff, if you think about it. You’re an intelligent person. Imagine I’ve locked you in a box, and the only way you have of interacting with the world is that you receive a slip of paper, and someone says, “Can you predict what will come next?” You predict what will come next, and then your memory’s wiped. You get another slip of paper. Imagine that was done a whole bunch of times, and then what comes out on the other end. They say, “Look at this essay that you wrote.” You might look at that and say, “This is awful. That’s not the essay that I would’ve written.” The process of repeatedly predicting something is just pretty different from how you would think as a writer to compose it and think it through. In particular, what would probably happen is that you’re a slave to your context. You might be answering some question about a particular field, so you draw on all the context around that. But the connection where all the substance is going to come from is, by its nature, a very unlikely one. You can do all the RL that you want to try to get better in some way, but what’s the thing that’s specifically upweighting and incentivizing making these unlikely connections when the vast majority of them aren’t the predictable next token that would come in there? So it might be the case that you just have this intelligence locked inside that box, but it’s a weird way of interacting with it. The thing I’m curious about is: do you ever get any fruit by questioning the premise of how tokens are generated? I don’t think it would be as simple as manipulating the temperature, but are there any things that you can do that take the existing level of intelligence but find the right ways of sparking those connections that unlock these sorts of things that we’ve seen? Or do you just need a little bit more intelligence, such that at the level of prediction, it’s predicting that it should be making that lightning bolt to another field? Dwarkesh Patel I think it’s more productive to reason, instead of architecture or even loss function, about data. We have diffusion models that do text, and the kinds of things they produce are not of a wholly different character. They’ve just not been explored as much. I think the more relevant thing is: what is the data on which whatever architecture or loss function you have is incentivizing you to produce? It does seem like they’re getting better. Forget about math. We did have a couple of examples of this kind of thing, but if you just look at why they’re getting better at being autonomous agents… They’re in an environment where they’re autoregressively producing the step that says “Let’s step back and do a search over the whole codebase,” and then “Let’s step back and assess my mistake,” is the thing that works. I assume what happened in the case of progress in science or maybe in math is you have frontier math-like problems. Mathematicians have specifically designed them because they require connecting together two different fields. I’m guessing there’s all kinds of clever, partially synthetic ways to make harder and harder problems like that that require these kinds of connections—for example, by eliminating assumptions and still requiring the AI to get to the answer—and then it doesn’t really end up mattering what the loss function is. It’s really about, can you come up with an environment that incentivizes this ability? Grant Sanderson It feels like you should be able to. I certainly can’t speak to the correct ways of doing that to unlock all this, but it would just be pretty surprising. Don’t you think it would be surprising if, over the next three years, there weren’t a lot more of those lightning bolts? Dwarkesh Patel I think this is an important thing to think about. We often think about how smart a single system is. And we don’t think about AIs having advantages that are more the result of other facts about them. So in this context, the key fact about them is that we can just parallelize and arbitrarily scale them. Whatever level of capability they have, it’s not just one idiosyncratic genius in the history of mathematics who makes a few connections and then dies in a duel. It’s universally applying that waterline across all problems that are accessible at that level of capability. This is among the many advantages that digital minds inherently have that we don’t think enough about. The other ones being that they can merge all their knowledge together—or at least that there will be techniques that allow this to happen—and that you can spawn off copies with identical levels of knowledge. This parallelization is quite an important property. I’d be curious about your predictions. Even if they’re not as smart as human mathematicians, the fact that for PR reasons the AI companies are just throwing billions and billions of dollars at this means that quantity has a quality all of its own. Grant Sanderson That seems in the right direction. If we take that conversation between Montgomery and Dyson at the IAS that suggests some connection between the Riemann hypothesis—or the Riemann zeta-function zeros—and random matrices, that feels like the kind of thing that you could try to automate. You have agents representing expertise in all these fields. We all know that an institute is smarter than an individual. The reason for having people all in the same geographic location is that you want those serendipitous conversations to happen. What does it look like to engineer those between agents? It’s interesting, because you point out that you can pool all your knowledge, but I really wonder if one of the advantages is that you can do the opposite of that. Sometimes when an AI is failing, it’s because it gets into a bad chain of thought and it’s really hard to get it out. So you say, “I’ll just start again.” Same deal with humans. Sometimes you start thinking about it in a certain way, and what’s required is to just back up. There are stories about people trying to prove something for a long time, and then at some point they say, “Hang on a second. What if I tried to prove that it’s impossible, or prove the opposite?” Unwinding your own context and going at it with a fresh mind… You could imagine systematizing that, or having multiple different agents deliberately given different pieces of context and trying to compare and contrast there. We don’t have the same level of manipulation on our own context. In this AI and math series, the first episode we’ll do will be about when they solved the IMO. I want to focus on one specific IMO problem that they failed on, which is one that a lot of very smart students failed on. Terry Tao also failed on it. People were very mad at the problem because they called it a troll problem. I almost don’t want to spoil it, because I want to construct the episode around leading someone in without their knowing that it turns out to have a simple solution. You can really empathize with what it’s like to be a student solving this. Basically, there’s a really elegant way of going down what you really feel like is going to be the solution based on the context of it being an International Math Olympiad problem. The character of the solution is really enticing, but it’s hard to prove that it’s the best. The reason is that it’s not. There’s this almost brain-dead solution that is the best. The relevance of that to the whole AI story is that for a human, what’s required to answer that question is to escape your context. Escape the context of being in the IMO. Escape the context of the way you’ve been trained to solve these contest math problems. If you just approached it like a brain teaser that I throw at someone off the street, they’d probably answer it well. You want the same sometimes for human research in other contexts, just being able to refresh your thinking and come at it completely differently. Of all the advantages that digital minds have, that might actually be one of them: a more systematic approach to refreshing your thinking. Spin off two agents, one who’s trying to prove it and one who’s trying to disprove it, one who tries it this way and one who tries it another. They deliberately have different contexts. I would be curious to see, if we’re having this conversation three years from now, how many of the significant results that make headlines have that character of basically erasing the context previously, trying a bunch of different things as opposed to merging the results of a bunch of different agents. Dwarkesh Patel It is incredibly interesting, because a common concern people have about AIs is this entropy collapse where they all think the same way, because they’re trained in similar ways. This is why they’re bad at writing. They go down the same path and have similar patterns of speaking and so forth. But maybe the key advantage AIs have is that you can systematically… It sounded like one of the reasons the unit distance problem conjecture took so long to be disproven was that people assumed the conjecture was actually true, so they were mostly trying to figure out ways to prove it. Maybe one of the key advantages the AIs will have is to increase the entropy by systematically trying out both the negation and trying to prove the positive of any given statement, or being able to systematically give different agents different biases. It seems like an important thing in the history of human science is that Einstein was really motivated by this bias that things should look the same in different reference frames. He had multiple other biases like these, but that one was very formative in his thinking. You can systematically survey a bunch of heuristics and see which ones are being productive on a given problem. Grant Sanderson So you would suggest systematically increasing entropy at the prompt level even though you have this inevitable collapse at the autoregression level? Einstein would be an interesting example, because he’s got this bias toward things being relative. He also has a bias toward “God should not play dice.” You want to make sure you don’t accidentally have all your LLMs be Einstein, because you might halt progress on quantum mechanics. Dwarkesh Patel Which goes to show you that there’s not a correct heuristic for science. You just need multiple independent research programs with their own heuristics. Grant Sanderson That feels like old-school software. As long as you’re able to describe that in some way. You have old-school software that amplifies that entropy. If you’re able to put a clear ontology to the distinct ways of thinking that you want to prompt, you explore that full ontology, and then each individual one runs off doing what it is. There’s a certain design question there about how exactly you describe the different approaches. The easy one is: are you trying to prove it or disprove it? The harder one would be to say, what are all the tactics you could take to prove this, and make sure you’re applying sufficient breadth to exploring them. 00:53:48 – Why real-world tasks don’t fit into RL environments

Dwarkesh Patel Obviously, AI in math is making much faster progress than everything else, and people point to the verifiability of the domain as the key reason this is happening. I think that’s one of the two important reasons, but people really neglect the other one. I’m outside the labs, so I don’t know what’s actually going on. This is a totally naive theory. A tangential question to why AI is making so much progress in math: why has it been so slow at computer use? A computer is very verifiable. Is my Etsy package coming? Is my event booked? These are extremely verifiable things to survey. What computer use lacks is grindability. Because websites have bot detectors—and it takes a tremendous amount of compute to run parallel rollouts—it’s very hard to run a thousand parallel rollouts of the same checkout flow on Amazon. You’ll get shut down by Andy Jassy. Grant Sanderson Him personally. He presses the red X on Dwarkesh button. Dwarkesh Patel Exactly. You could try to build clones of every single website, but that’s very labor-intensive and slows you down. The reason you currently need to do so many parallel rollouts to learn a skill with deep learning is that we haven’t solved sample efficiency. Grant Sanderson Sucking supervision through a straw, as Karpathy says? Dwarkesh Patel Exactly. Of course people are working on many different techniques, but fundamentally there’s this big constraint in the way we train AIs. With code, you can containerize a given level of progress in a repository and then spin out hundreds of parallel containers and say, “Try to implement this feature,” and it’s totally deterministic. Because it’s deterministic, you can solve the credit assignment problem because you know that whatever caused this rollout to succeed and this one to fail, the diff is the thing that worked. If you have situations that are starting off at different starting points, this credit assignment problem becomes much harder to solve. Most things in the real world are very hard to containerize in the same way. Coding and math are exceptions to this rule. But if you’re trying to figure out how to build a new business that succeeds, or how to go trade in the markets for a day and make money, the fact that you have to interact with the real world and things change day after day means that you can’t keep replaying and grinding and farming the simulator. Math, of course, is the exception, and I feel like this is an important driver of progress in this domain and also in coding. It’s not just verifiability; it has to be grindable. The third reason people point out that AI is making fast progress is they focus a lot on Lean and formalization. Again, I have literally no idea what’s going on in the labs. I feel like Lean just doesn’t matter that much for the current level of progress in AI. Why is AI able to disprove the conjecture about the unit distance problem? They released the chain of thought, or at least a rewrite of the chain of thought. It didn’t have any Lean in it. I think the process-based supervision that Lean provides, where you know each step is correct, seems less relevant than just having this grindable outcome that is verifiable. Grant Sanderson It’s an interesting point about grindability mattering more. Naively you might think Lean provides something unique for math because you’re able to see if it can prove it. You have old-school software that can tell you yes or no, and you use that as your VR. What would corroborate your point is the initial attempts. Again, I’ll circle back to the IMO. Initially, DeepMind basically does that. Everything is in Lean, and then the next year it’s all in natural language. So to your point, it’s not needed. I do think there’s a yet-to-be-explored benefit of that formalization domain, which is that at the moment you still need a human reviewing that counterexample to the unit distance conjecture to say, “Looks good.” That provides a certain bound on how endlessly explorable things are. If you consider AlphaGo or AlphaZero-style systems, they’re off in their own universe playing a bunch of Go and exploring themselves, potentially going off the rails of what any human needs to look at, but they still have this automated verifiable reward. It’s not just that you can do RL on that. It’s also that you basically never have to check in, and you can just pour compute at them exploring the universe of Go. What stands to be interesting—maybe this won’t pan out, but the jury should still be out on whether it’ll yield anything—is that with Lean, you could imagine having a basically endlessly running program that’s constantly trying to extend Mathlib. Mathlib is this GitHub repository that’s basically all of math written in code. It’s very far from all of math, but they want it to be all of math. It’s written in code where you can ask, “Is this proof correct?” It’s very labor-intensive to write these proofs. There’s a whole subcommunity around it. But you could imagine having an AI where you say, “Simply try to extend Mathlib.” Maybe it’s a fork of it so that it doesn’t have trash in it, because people have a certain taste for what they want to be in there. So you have your fork of the pure AI Mathlib, and it just goes and it doesn’t stop. It doesn’t need anybody to check in on it. It could just keep going. It might come up with its own conjectures. It might come up with its own theories and different definitions. Maybe many of them are useless, but it just has this infinite tree that it can grow out. That’s a very unique thing that math has that nothing else has, where you could press go and just pour compute at it, look away for ten years, and then come back and say, “What do you have?” There’s going to be something. Then there’s a question: is it useful or not? How do you suss that out? That’s just an interesting thing to be able to do. It would be very surprising if that didn’t yield some sort of interesting mathematical insight from it. There are two different ways that Lean is important in this story. The first one is how you could let go, not even check in, and progress will be made. You can do that with Go. I don’t think you can do that with natural language math. Dwarkesh Patel That’s very interesting. Did you see Karpathy’s auto research idea? He wrote this one Python file that does basic LLM training, and then had a repo where LLM agents would try to make modifications to the file, and if it sped up the speed run, the modification stays. Eric Jang, who came on to explain how AlphaGo works, did a similar thing when he was trying to build a very strong Go bot. He had interesting observations. It’s really good at running an experiment and going down that path, but it’s bad at stopping at dead ends and doing extremely parallel things. Anyway, this will probably change in the future. It’s very interesting to think about what it looks like in the limit. This is fundamentally what the human institution of mathematical research is. It’s a library extended in interesting and useful ways. This way you don’t have any outcome-based supervision. There’s no outcome that you’re trying to incentivize, but you have a process. You know the steps are correct, you just don’t know if it’s going in an interesting direction. Grant Sanderson If you were doing that, you don’t want to completely go off the rails and do a random walk through the space of logic. You’d probably want some supervisor model that’s trying to provide heuristics on whether it’s useful or not. You know people are working on it. That’s one of those “five years from now” things where I’d be curious to get the future version of us talking about it. Maybe that goes nowhere, but Terry Tao was talking about one research project that tries to exhaustively search the space of possible algebras. You could imagine different axioms that you apply to algebraic systems. When we come up with group theory, there’s a certain axiom system that looks like arbitrary rules unless you know the motivation. What if you tried all of them? Do any of them yield useful things? The vast majority of them are just trash in some way. It all collapses to no interesting results. But every now and then, there would be this little island of a completely different type of axiom system that at the very least seems rich in terms of the number of theorems that can come out of it. That’s bread and butter for what you would imagine automated provers being good for, exploring that space and seeing which one of them turns out to be something. Maybe one of those islands actually turns out to be something you can retroactively put motivation on, to say this is the kind of structure it’s trying to get at. In the same way that you could imagine looking at the axioms for a group, not knowing that it’s about symmetry, but you retroactively realize this is very relevant to studying symmetry. You could imagine results of that flavor, but instead of just exploring possible algebra systems, it’s exploring all possible logical consequences of any kind of axiom. Dwarkesh Patel On the point about whether you can provide process-based supervision without Lean, DeepSeek had their DeepSeek Math model. They released a paper on how they trained it, and it was quite interesting. The problem with natural language proofs is you don’t know if it’s correct or not. They have a verifier, and the verifier is trained by a meta-verifier that makes sure that for all the problems they’re training this model to solve in the art of problem-solving, the verifier is giving good feedback. It works. It’s interesting that natural language verification with some sort of meta-verification seems to work so far in the published literature. It also seems to work in the published products that we’re using. If you look at coding agents, they’re getting better and better at writing clean code and refactoring code. I’m sure there are process-based “LLM-as-a-judge” systems providing taste and saying, “Is this a clean way to write this function? Are there duplicates of the same kind of modular forms?” That should also work for mathematics, right? Grant Sanderson It seems more plausible for math than anything else, even if you’re only working in natural language, that you could trust a verifier. You and I were talking earlier about why they’re bad at writing. They seem to be good judges. If I give them two essays that students wrote, they’d be able to say which one is more accurate and insightful. So why can’t you just have a verifier saying, “Is this a good piece of writing or not?” Maybe the ultimate failure there is that even if they’re good at discriminating between a B essay and an A essay, they’re not actually good at discriminating between an A essay and a thing you actually want to read, something that would be followable on Substack and insightful. They actually end up preferring uninsightful pieces of writing. On the math front, the step to simply know if a proof is correct or not lends itself to an automated verifier, even in natural language. You could probably still make a ton of progress. I still like the tree of logic out of Lean, just in that you can really go off the rails. There’s no constraint on the previous way things had been phrased before. Everyone talks about move 37 in AlphaGo. What is the thing that lends itself to going outside the prior heuristics? It seems productive to have a disconnection from the rest of the world in that exploration, as a complementary research pursuit to the natural language math front. The other relevance of Lean would be, let’s say you have your pure natural language RL environments and a pure natural language set of proofs. People say, “Proceed, AI mathematicians,” and they generate ten papers a day. If there’s any error rate to that at all… Alex Kontorovich has talked about this. It becomes insufferable as a mathematician. Every single time you see one of these, you don’t know if it’s worth your time. Even if 99 out of 100 are right, I don’t know if it’s worth my time because it’s really labor-intensive to find what that error would be. It’s really frustrating to spend all your time on a paper that was trash. Having something that’s able to give you that green checkmark that says, “Even if this is going to be complicated to understand, even if it’s going to be a pain, you at the very least know it is correct,” every other field would kill for that. Math has that. If the models are also able to take their natural language proofs and formalize them, that seems huge. Every field would love to have something like that. So I think you’re right that Lean is maybe overrated regarding its importance as a VR environment for progress in math generally. But I definitely wouldn’t write it out of the story. Dwarkesh Patel I also love this extension of Mathlib as a metaphor for what’s going to happen to our civilization pretty soon. For millennia, humanity has built this corpus of knowledge and understanding, and everything that we have is now distilled into these models. At some point, the models will just extend that arbitrarily. 01:07:07 – Good writing requires theory of mind that AI still lacks

Dwarkesh Patel By the way, on the writing front, I have a theory of why writing is making worse progress than these other domains. One reason is what you said, that they’re bad at judging not only A versus B, but they get totally derailed by B*, which is this shitty essay that hits all the bells and whistles that A is supposed to hit. The reward hacking thing just goes off the rails. But the other important thing is that writing is not modular in the same way that code and math are. You can write a function many different ways, and they do the same thing. Of course you want it to be clean, but at the end of the day, if it works, it works. Same with lemmas in mathematics. You can have some end product that’s different from the way it’s produced. Code is the thing that produces some end product, and you want a functional end product. Whereas in writing, the end product is directly the thing the AI is producing. Each paragraph, sentence, and word matters because that is the substance. It’s not some separate thing produced out of the writing. It can’t be slop in the way that code can be slop and still produce the outcome you want. Grant Sanderson But you were just pointing out how we’ve actually gotten much better at agents writing not just functional code, but clean code. Why is it not the case that the same progress that lets you go from merely functional to a clean and mergeable PR also results in clearer writing? Dwarkesh Patel That’s a good point. Also, has it not? I agree there are many ways in which they’re terrible writers. But for a lot of writing I consume, I find it’s better to just copy-paste it into an LLM and say, “Explain this to me.” The explanation will be better than the thing produced by the human. It’s funny that we say these are such terrible writers, and yet my revealed preference is to have an LLM explain it. Even when I’m talking to a human expert live on a call, if it’s a piece of knowledge that only they have that’s not encoded in the distribution, I want them to explain it to me. But if in order to understand that, I need to understand a more basic concept, I would prefer if it were socially acceptable for me to just say, “Let’s pause here. I’m just going to ask an LLM how that works, and then we can come back to your special piece of knowledge.” Grant Sanderson That’s distillation, an explanation. If I’m thinking of your quality as an essay writer—if I give you a book to read and I want a book report—I might believe that the LLM gives me a better book report. But what people are really getting at when they say it’s bad is, what is writing? It’s not just the distillation of preexisting ideas. It’s not just how you explain clearly, because they are good explainers. It’s about what the insight is. This is where autoregression is a very weird way to generate things. When you’re writing, you sort of know that in order for it to be good, you have to have an element of the unpredictable. It’s not just increasing the temperature in your mind. It’s knowing exactly the correct point when you want to make an unpredictable move, and that that’s going to be what’s more insightful. Even if it’s better at explaining a preexisting thing, what generated that book that you wanted distilled in the first place? It wasn’t an LLM that generated it and you just needed it. It was some author who, through a lot of exploration of ideas in the world, decided what aspects were interesting and what ways of presenting it formed a coherent, well-motivated narrative. They put that all together in some way. If they’re a good author, you would probably err on the side of reading their book instead of the distillation. Still, what makes it worthwhile to explore at all in the first place and want to upload it at all? It’s that side of it that people cite when they say LLMs are bad at writing. It’s that element of unpredictability, of deliberately choosing something novel that is very directly contradictory to the way things are typically produced. Dwarkesh Patel That’s a good point. I think they’re also really bad at building really good mental models of people, which is a very important skill in writing. Andy Matuschak and another collaborator, whose name I’m forgetting right now, did an interesting report where they tried to teach LLMs to write good spaced-repetition prompts. I really like this because even though it seems like a totally random skill… It’s just like, people are talking about recursive self-improvement in a year, and we can’t get these things to write good flashcards. What’s going on there? They tried many different kinds of techniques, and they’re sophisticated people. They tried to RL open source models. They tried all kinds of things, including chain of thought and a big prompt they sent to the best closed source model. The key constraint, it seemed to me, was that writing a good card is about projecting somebody’s mind in three months. What is the way in which they’ll associate the question? What kind of answer will they be thinking at that moment? Is the elicitation that inspires the detail you actually want to take away from the passage you’re trying to make cards about? I think writing is similar to this. If you’re writing something, the reason it’s such an enervating process that takes so long is that with each word or each sentence, you have to be thinking: what is happening in my reader’s mind right now? Even if I flip the phrasing around so the end phrase goes to the beginning and this is the first image that comes to your mind before you read the rest of the sentence… Maybe autoregression is bad at that. This is maybe a more diffusion-like property of considering the whole rather than going sentence by sentence. But also I think that requires a lot of mentalizing, which these models weirdly struggle at. Grant Sanderson It’s an interesting question. Is it weird that they struggle at that? I might butcher this. You know how you cite studies that you once read and maybe the study wasn’t real? There’s one very memorable one. Let’s say you want to quiz people’s EQ. You show a flashcard of someone’s facial expression and someone is trying to describe that emotion. There are really good tests online that have a face and then four possible emotions. It’s surprisingly hard to describe exactly the correct emotion, but you also get the sense there really is a correct answer. If you try this with people in your life, you’ll notice that the ones who are pretty plugged in socially do really well on it, and the ones who are a little bit more left-brain don’t. That is a kind of test you can do. I vaguely remember an experiment to this effect where they took people who had freshly gotten Botox, and they did a pretest and a post-test. Post-test, they were just much worse at reading people’s expressions. That feels weird. Dwarkesh Patel Wait, they got Botox? Grant Sanderson The person taking the test. You do the test, and then you go and get Botox and your face is all frozen, and now you’re worse at understanding the emotions of what you see. The thought is that part of understanding the emotion you’re looking at is doing it yourself. At a facial level, you’re moving your face muscles. You see that, you mimic that, and you’re like, “Oh yeah, that’s anxiety,” at some very subconscious level. So in that sense, if it is the case that models have bad theory of mind, sure, they know everything because they’ve read what everyone wrote. But at the level of actually being able to put themselves in your shoes in the same way that my face muscles are mimicking your face muscles—that’s what helps me understand how you feel—it’s not surprising at all. They don’t have face muscles. Their brain works completely differently. It’s like an alien trying to empathize. How could it have theory of mind? It would be this very emergent thing to have. Whereas we can just plug it into our own minds. We’ve got the ready-made hardware to just place it in. From that lens, it’s not that surprising. 01:16:02 – Why learning will still depend on human curation

Dwarkesh Patel What advice do you have about using LLMs to learn? As I was describing, for a lot of well-known concepts, I find them very helpful. But often, just a couple of messages further down, I’m trying to understand something, and they’re so confused themselves that they’re confusing me. They don’t explain it the right way. I know that talking to the right human could clear up my confusion in three minutes. More and more, we’re going to want to use these things to learn. People talk a lot about education and representation stuff. Have you noticed ways to use them more productively to understand concepts? Grant Sanderson I’m curious to hear your take on this. I’ll give mine. Even pre-LLM, I feel like a relevant insight in learning was recognizing who matters more than what. My advice to any college student when they’re choosing what courses to take: care a little bit less about your preexisting interests, because they’re kind of arbitrary right now, and care a little bit more about whether the person teaching it is a good educator and someone you resonate with. In choosing what books to read, who the author is maybe matters more than whether it’s a prior interest. If there’s a book you’ve liked before, read what else that author has written rather than reading another thing on that subject. I’m getting to LLMs on this. There’s a difference in feel for trying to learn something from a Wikipedia page versus, if it’s a philosophy topic, going to the Stanford Encyclopedia of Philosophy. Or if it’s a math topic, you go to the Princeton Companion to Mathematics. The difference there is the articles are deliberately written by one individual who tries to actually craft a motivation around it. Whereas on Wikipedia, it’s this local minimum that’s reached where every sentence has to be correct. In a good exposition, you care a little bit less about correctness on the way. You can deliberately craft things that are a little bit wrong that you correct along the way, which gets edited out in a crowdsourced environment. LLM explanations feel to me at the moment a lot like Wikipedia, which is to say, amazing. Imagine a world before Wikipedia, how long it would take to find and suss everything. But nevertheless, what’s the most useful part of a Wikipedia page? It’s often just the references at the bottom. You look at the key references, and you go to them, and you read them. Sometimes that gives a much better overview. So often I like to just ask an LLM, “Who should I read?” Maybe I can even give some specifics on ways I want to learn. I actually got gaslit by this once when I was trying to learn about semiconductors or something. I felt it was a very visual topic, but all the resources were text. I asked, “Is there a well-visualized video explaining the concepts you’re getting at?” And Claude said, “Yeah, here’s a couple,” and the top one was like, “Here’s one from 3Blue1Brown”. I’m like, “I can guarantee that there’s not.” It was an actual video, an actual link, but it had just misattributed someone else’s. It was good. I had a much better experience clicking over and watching it to learn rather than trying to proceed forward with questions there. In that sense, I’m basically using it like a very souped-up version of Google to zero in on the right human-written resource. What about you? You engage with these a lot. What’s the best way to use them? Dwarkesh Patel I think you put your finger on it. The most productive learning sessions I’ve had are when there’s some artifact that a human has produced—whether it’s an article, a book, or a video—that organizes the relevant concepts in the correct way. It builds up the motivation for why the next idea would be relevant to solving the next problem you’d encounter, and the next idea, and the next idea. Then you use the LLMs to just do a little bit of pruning around this branch that the book has identified. I was actually going through—I think you might have recommended it—Steven Strogatz‘s textbook on… Grant Sanderson The chaos one? Nonlinear Dynamics and Chaos? I love that book. Dwarkesh Patel Yeah, I was going through it, and it was bliss. It was like your videos in book form. It was super fun. The way I was learning it, I’d have his university lecture on one-third of the screen, that part of the textbook on another third, and an LLM on the last third. I was actually thinking, if I were back in college and watching this lecture live, it would totally go over my head. These kids must be really smart, because I’m pausing, reading the textbook, talking to LLMs, and then restarting again. But with him curating the right order to understand concepts and the right problems to motivate understanding them… Another thing LLMs are really bad at. Something a really good human can do, when you ask a question, a human can say, “Actually, you’re not really thinking about this topic the correct way. The question you want to be asking, the correct way to organize these concepts, is X.” An LLM just can’t really do that. Grant Sanderson It’s a little too placating. This is ultimately that sycophantic behavior where it’s very, “Oh, what an insightful question.” You want to strip that down. That’s a good point, and I think it cuts to theory of mind a little bit, recognizing that asking a certain kind of question reveals that the student’s mental structures are not the same as the explainer’s. Sometimes people do this to a fault. With a really good teacher, let’s say you have a middle school math classroom. If a student asks a question that suggests they’re thinking about it in a different way, it’s actually really hard to take that seriously in the moment and ask, “Hang on, could you get to a right answer with that?” before you say, “Instead of that, let’s do this.” The really good teachers are able to jujitsu the creative way the student was thinking about it and bring it in. LLMs aren’t doing that. They aren’t reframing your question. Instead, they kind of run off. At the very least, it feels like there are three levels here. An LLM is at one, a good explainer is at another, but the A+ explainer is the one who can jujitsu your way of thinking and say, “That’s where that’s useful.” Maybe there is a cycle all the way around where, five years from now, the LLMs will be doing that, but in a better way. Dwarkesh Patel What is your recommendation to students who I’m sure email you this question all the time: “I was curious about doing mathematics. I’m really passionate about the subject, but seeing all the progress AIs are making, I don’t know if it makes sense for me to pursue this as a career.” This is relevant not only to people in mathematics, but to anyone noticing that their field is getting productivity gains from AI. Coding is very adjacent to this. What advice do you have for people? Grant Sanderson I wouldn’t trust any advice that I give. That’s how I’d couch it. But even pre-AI, it feels very important for any job you’re going to go into to really understand… If we’re talking about a job—not being a gentleman-scientist engaging with the math world or something—you should understand where the money is coming from, what value you’re actually adding, and the connection between those two. A surprisingly small amount of thought is put towards that, especially by students. They’re in this environment where they probably want to go into math because they’ve always been good at it. They’ve been rewarded in life for proceeding through the next hoop correctly. When they think they want to be a mathematician, it’s because they think it’s a way to continue engaging with that. They think, “Where do people get to do this?” rather than thinking, “What value am I adding to other people, and to what extent is that the reason a salary is flowing in my direction?” It’s actually quite different in different cases. In some cases, it’s a very prestigious mathematician, and their presence at a university lends a certain brand value, which is why the university wants them. In some cases, an NSF grant is given because of the public good belief we have around basic science. You’ve got an institution around that, and a whole bureaucracy acting as a proxy for what we think that public good is, with a whole song and dance around how to make them correctly predict that your progress will be in the spirit of that funding. Sometimes it’s just straight-up teaching. People like to send their kids to an institute that has experts teaching them. You provide brand value by being an expert, and direct value by being a teacher. Regardless of whether AIs are proving theorems or not, or whether we’re talking about 2016 or 2026, that is something not enough students thinking “I want to be a mathematician” consider. I think it’s worth thinking about. For me, I wasn’t necessarily thinking about it, and I stumbled into a career path where math exploration can be monetized as entertainment. I stumbled into that and I’m very grateful I did, but it was an accident. It wasn’t deliberate. I could have avoided relying on serendipity and done it a little bit more by design had I been thinking critically about it. To your question—if we have almost-automated theorem proving, and let’s say they’re also really good explainers so you even get the human understanding—I think a lot of the social role that mathematicians serve actually doesn’t change that much. As a public, we still feel there’s value to basic science, and we trust the judgment of mathematicians to determine where their time is best spent. The prestige comes from within that community. It’s other members saying that a result is really good, more than the grant writer really understanding algebraic number theory to understand it was a good result. There’s going to be an inner culture of what constitutes valuable contributions. Maybe it shifts away from theorem proving and towards good definition writing. Maybe it’s that museum curator idea. But you’re going to have that same community as long as society as a whole is still valuing the premise of basic science. And if we’re in the abundance world that AI brings, there’s probably more funding in that direction in some sense. On the side of prestige to institutions for who their lecturers are, I actually think teaching is one of the most stable post-AGI jobs that there is, because it’s so relational. This is where parents want to spend their money if they have an abundance of wealth: on good teaching and good educating. It goes so far beyond explanations. Even if LLMs are good explainers, the thing that a teacher is doing is such a social, coaching, mentor-type thing that that’s probably one of the most stable careers that’s going to exist over the next fifty years. Insofar as a lot of mathematicians’ roles overlap with that, as the prospective student going into it, you could lean into that. I actually think a lot more students should think about and give credence to the idea of being just a math educator and the value that can serve towards the next generation. I’ll couch again that I don’t think I’m the one to say, “Here, prospective young mathematician, here’s how you should think about the future,” because I’m a YouTuber. I’m not in the institution that they’re thinking of going into, so I’m speaking as an outsider looking in. But it feels like generally good, universal advice: know where the money is coming from, know where you plug into that. And if you’re just asking those questions, you’re actually already steps ahead of all the other fledgling prospective mathematicians. Dwarkesh Patel In fact, think about the crazy world where, within five or ten years, the AIs are coming up with not only solutions to the Millennium Prize problems, but totally novel problems to be solving in the first place, novel mathematical fields and objects and stuff. It is in that world where, first of all, there’s a ton of abundance. Two, the thing AI minds will have gone furthest in, where they will have seen furthest beyond our horizons, will be mathematics. There will be so much demand for, “What have the AIs seen? Can you explain it to us?” In that world, if there are any jobs whatsoever, surely distilling what the AIs have learned will be one of them. Grant Sanderson Also, it’s funny because all of this presumes that it’s useless. We’re not talking about the actual practical applications of what math is being done. Insofar as there’s any economic utility to it, you would imagine that the people who understand it and are able to make the decision of where it should point actually have a lot more economic value by being able to make that judgment as a curator and point this behemoth of new math in a useful direction. Suddenly, that’s a much more levered move to make than it had been previously. Dwarkesh Patel Can I ask you about that? Obviously, one question for AI for math is not only can it do it, but is it any good? Or is it good for anything? You were describing all the ways in which, with group theory, we’re trying to figure out random facts about the roots of different kinds of functions, and now there are all these different applications that are practical across many different fields. Do you have some sense of whether, if we just totally get to a place where the field of human mathematics is accelerated 10X or 100X and some crazy shit happens, or are we just going to be bottlenecked by other fields? Grant Sanderson I think there are some fields that probably will. It’s super spiky. With progress in algebraic number theory, it feels unlikely that that then unlocks something. But I remember talking to this mathematician who does more dynamics and PDE-solving type stuff. He was referencing that his group had some ideas. Let me see if I summarize this right. It’s like the way Boeing would make planes is that they’d make it, do a bunch of tests, and they had to disassemble it and reassemble it based on those tests. His group essentially had some insights on how to do more in simulation such that you don’t have to deconstruct and rebuild it. It saved Boeing billions of dollars or something, and then they just started funding that group. That’s much more obviously application-adjacent, because PDEs just are that. Progress in that domain, you would imagine actually does unlock some things. I don’t know if it’s these step changes, but maybe it’s more on the side of engine design becoming a little bit more fluid, or coming up with the right wing shape instead of running a whole bunch of complicated CFD. Maybe you’re able to speed up your CFD simulations because certain pure math insights make those more efficient. I bet you’d just see a lot of great incremental improvement there. It seems less likely that the massive breakthroughs in math immediately turn into this massive economic breakthrough, like you solve the Navier-Stokes problems, and then that unlocks an ability to simulate more things. But you probably will see, at those fringes, some meaningful leakage out of the pure math insights into other things. There’s a ton of people working on things like AI engineering, physical engineering, and material science. You have to imagine they’d be in a good position to look at the AI math insights and decide whether they’re relevant in some way or not. It’s another one of these things where I’m not going to sit here and put a flag in the sand predicting that there will be. But it’d be a little bit disappointing and a little bit surprising if there weren’t, over the next five years, economically valuable improvements made that were directly referable to the AI progress in math. It would just be disappointing if it was just taking down a bunch of Erdős problems and none of them were doing any of the math that actually directly touches the physical world. Dwarkesh Patel To your point about how a lot of the history of mathematics was about building up these piles of concepts and connections. Sometimes the piles connect with each other, or you discover an application somewhere else. At the very least, you just build up this huge pile. Then as broader progress in society happens during the singularity, when we get to the industrial part of the singularity, you just have all these different ideas that hopefully are useful in other parts of the world. Grant Sanderson As I said, one of the interesting things about what’s happening is it causes people to step back and ask, “What is math?” Maybe one of the awkward conclusions will be revealing that it’s just become wholly useless. The kind of questions being asked have become so divorced from things that are physically applicable that that’s one of the things mathematicians have to come to terms with. Everyone will look and say, “Hang on a second, weren’t you guys supposed to… If there’s 10X progress there, why aren’t we seeing it over here?” And then mathematicians are like, “Ugh.” Every time we wrote those grant proposals and said, “Trust us, the elliptic curve progress is going to help with cryptography,” it shines a light on the fact that maybe it doesn’t. So that’s one possibility. Dwarkesh Patel Grant, this was super fun. Thanks so much for doing it. Grant Sanderson Absolutely. My pleasure.