Great article. I do, however, have some significant disagreements - I'm pretty much on board with the mainstream, EA orthodoxy. Beginning with your worry about nukes, I agree that nukes are a collective action problem - that's one reason why it would be a bad idea to unilaterally disarm. But we can reduce nuclear risks without doing that. Some ways to reduce nuclear risks include shrinking the nuclear stockpile, adding more safety, etc. A lot of past nuclear risks have come from accidents - for example, there was a time when we dropped a nuke in one of the Carolinas and 5/6ths of the switches malfunctioned - if the last one had, it would have nuked a bunch of the state.
On the AI stuff, I broadly agree that AI won't objectionably displace jobs. But I think that, while there's some uncertainty about AI, we have decent ways of knowing things about AI. For example, my credence in AGI in the next century is quite high - based both on expert projections and the most sophisticated report, conducted by Katja Grace, concluding that. We also know that, if AI is much smarter than us, it has a high chance of being dangerous, especially if we can't control it. So that makes it prudent to try to control it. (There's obviously much more to be said about AI, but I don't want my comment to be 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 words :) ).
I'm not sure why one has to deny that things will get better to be a longtermist. One of the reasons I am a longtermist is because I think things will get better - and I want there to be a lot of people living that better life.
Infinite Sets are a source of interesting paradoxes. Some of the questions have puzzled mathematicians for as long as they have been thinking about them.
"What is larger," wondered Galileo Galilei, (definitely not the first person to wonder about such things), in _Two New Sciences_, published in 1638, "the set of all positive numbers (1,2,3,4 ...) or the set of all positive squares (1,4,9,16 ...)?"
For some people the answer is obvious. The set of all squares are contained in the set of all numbers, therefore the set of all numbers must be larger. But others reason that because every number is the
Galileo concluded that the totality of all numbers is infinite, that the number of squares is infinite, and that the number of their roots is also infinite; neither is the number of squares less than the totality of all the
numbers, nor the latter greater than the former; and finally the attributes "equal," "greater," and "less," are not applicable to infinite, but only to finite, quantities.
We haven't made a lot of improvement to these conclusions since Galileo's time. The major enhancement was done by Georg Cantor which has at its subject the 'cardinality' of sets. For finite sets,
the cardinality is the number of items in the set. You just count them -- what you get is what you get. This is problematic when you know that the set is finite, but very large, as in 'you will be dead before you count off the last number'. Better hope there is a shortcut to working things out. You don't have to count out 'the set of all positive integers less than 2**10000' to find out how many there are. But the set of all primes less than the same?'
Infinite sets also have a cardinality. (This is what Cantor worked on.) A substack reply is definitely not the place to go about discussing Cantor's groundbreaking work on the cardinality of different infinite sets,
but if you are interested, look up 'the cardinality of the continuum' and 'the cardinality of infinite sets'
looking for.) There are webpages about this to suit every level of mathematical sophistication, from the grade school student who only learned what a set was last week, to mathematical post docs working in the field.
It's fun to think about. But I digress. The bottom line is that we know a good number of properties of certain infinite sets. The smallest sort of infinite set -- and we have a proof that this is the smallest, too -- is the countable infinite set, which has a cardinality called 'aleph-null' in the jargon. The set of all natural numbers is countable. So is the set of all even numbers. Or all squares. Or all multiples of 5. There are other sorts of infinite sets -- the set of all real numbers being an example -- which aren't countable. Their cardinality isn't aleph-null. And _sets with the same cardinality have the same size_. If you know that two infinite sets have the same cardinality, then you know they have the same size, even if you don't know what it is.
Which brings us to the utilitarian's favorite hobby horse -- trolley problems. Each person tied to the tracks represents an infinite set of possibilities of things that that person could do, in the future, if
the train doesn't run over them. So, while we may not know what the cardinality of the set of set of missed choices _is_, that a person about to be run over by a train _is_, but in the sort of hand-waving, thought-experiment way we think about such things, it seems reasonable enough to assume, for the purpose of a thought experiment, that all people have the same cardinality, which is in some way a measure of their utility (in the utilitarian sense).
Note: I am most definitely not saying that I can prove any of this. This is 'Cheers! Here is a beer, let us sit down and amuse ourselves thinking about math and philosophy' time. But modelling people mathematically as infinite sets of possibilities with the same cardinality seems a fair approach. And with it, things do not look very good for the Utilitarians in the philosopher's pub. When they reason that killing 5 people is 5 times as bad as killing only 1, they are reasoning in precisely the same way as the people who believe that the set of all numbers must be 5 times larger than the set of all powers of five.
Thus, the good mathematical argument is thus with the virtue ethicists and the deontologists, who have been saying, all along, that the badness of murdering people by running trains over them is not something that you can calculate.
Please remove the excessive amount of 000000. The computer reading then out is crazy annoying and I had to skip a bunch. Other than that good article, still reading.
Great article. I do, however, have some significant disagreements - I'm pretty much on board with the mainstream, EA orthodoxy. Beginning with your worry about nukes, I agree that nukes are a collective action problem - that's one reason why it would be a bad idea to unilaterally disarm. But we can reduce nuclear risks without doing that. Some ways to reduce nuclear risks include shrinking the nuclear stockpile, adding more safety, etc. A lot of past nuclear risks have come from accidents - for example, there was a time when we dropped a nuke in one of the Carolinas and 5/6ths of the switches malfunctioned - if the last one had, it would have nuked a bunch of the state.
On the AI stuff, I broadly agree that AI won't objectionably displace jobs. But I think that, while there's some uncertainty about AI, we have decent ways of knowing things about AI. For example, my credence in AGI in the next century is quite high - based both on expert projections and the most sophisticated report, conducted by Katja Grace, concluding that. We also know that, if AI is much smarter than us, it has a high chance of being dangerous, especially if we can't control it. So that makes it prudent to try to control it. (There's obviously much more to be said about AI, but I don't want my comment to be 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 words :) ).
I'm not sure why one has to deny that things will get better to be a longtermist. One of the reasons I am a longtermist is because I think things will get better - and I want there to be a lot of people living that better life.
Infinite Sets are a source of interesting paradoxes. Some of the questions have puzzled mathematicians for as long as they have been thinking about them.
"What is larger," wondered Galileo Galilei, (definitely not the first person to wonder about such things), in _Two New Sciences_, published in 1638, "the set of all positive numbers (1,2,3,4 ...) or the set of all positive squares (1,4,9,16 ...)?"
For some people the answer is obvious. The set of all squares are contained in the set of all numbers, therefore the set of all numbers must be larger. But others reason that because every number is the
root of some square, the set of all numbers is equal to the set of all squares. See 'Galileo's Paradox' https://en.wikipedia.org/wiki/Galileo%27s_paradox .
Galileo concluded that the totality of all numbers is infinite, that the number of squares is infinite, and that the number of their roots is also infinite; neither is the number of squares less than the totality of all the
numbers, nor the latter greater than the former; and finally the attributes "equal," "greater," and "less," are not applicable to infinite, but only to finite, quantities.
We haven't made a lot of improvement to these conclusions since Galileo's time. The major enhancement was done by Georg Cantor which has at its subject the 'cardinality' of sets. For finite sets,
the cardinality is the number of items in the set. You just count them -- what you get is what you get. This is problematic when you know that the set is finite, but very large, as in 'you will be dead before you count off the last number'. Better hope there is a shortcut to working things out. You don't have to count out 'the set of all positive integers less than 2**10000' to find out how many there are. But the set of all primes less than the same?'
Infinite sets also have a cardinality. (This is what Cantor worked on.) A substack reply is definitely not the place to go about discussing Cantor's groundbreaking work on the cardinality of different infinite sets,
but if you are interested, look up 'the cardinality of the continuum' and 'the cardinality of infinite sets'
see: https://en.wikipedia.org/wiki/Cardinality_of_the_continuum (more to figure out what to start
looking for.) There are webpages about this to suit every level of mathematical sophistication, from the grade school student who only learned what a set was last week, to mathematical post docs working in the field.
It's fun to think about. But I digress. The bottom line is that we know a good number of properties of certain infinite sets. The smallest sort of infinite set -- and we have a proof that this is the smallest, too -- is the countable infinite set, which has a cardinality called 'aleph-null' in the jargon. The set of all natural numbers is countable. So is the set of all even numbers. Or all squares. Or all multiples of 5. There are other sorts of infinite sets -- the set of all real numbers being an example -- which aren't countable. Their cardinality isn't aleph-null. And _sets with the same cardinality have the same size_. If you know that two infinite sets have the same cardinality, then you know they have the same size, even if you don't know what it is.
Which brings us to the utilitarian's favorite hobby horse -- trolley problems. Each person tied to the tracks represents an infinite set of possibilities of things that that person could do, in the future, if
the train doesn't run over them. So, while we may not know what the cardinality of the set of set of missed choices _is_, that a person about to be run over by a train _is_, but in the sort of hand-waving, thought-experiment way we think about such things, it seems reasonable enough to assume, for the purpose of a thought experiment, that all people have the same cardinality, which is in some way a measure of their utility (in the utilitarian sense).
Note: I am most definitely not saying that I can prove any of this. This is 'Cheers! Here is a beer, let us sit down and amuse ourselves thinking about math and philosophy' time. But modelling people mathematically as infinite sets of possibilities with the same cardinality seems a fair approach. And with it, things do not look very good for the Utilitarians in the philosopher's pub. When they reason that killing 5 people is 5 times as bad as killing only 1, they are reasoning in precisely the same way as the people who believe that the set of all numbers must be 5 times larger than the set of all powers of five.
Thus, the good mathematical argument is thus with the virtue ethicists and the deontologists, who have been saying, all along, that the badness of murdering people by running trains over them is not something that you can calculate.
Please remove the excessive amount of 000000. The computer reading then out is crazy annoying and I had to skip a bunch. Other than that good article, still reading.