Most of the time when you're working on a project, you can test it out, make changes, and iterate as many times as you want. And you can do this when the project is yourself too... kinda...
If your modification makes it so that you are less capable of self modification, want to self modify in new directions, or in any way fucks with the part of you that does self modification, then "you" might not get a chance to fix it.
There are reasons to think that this is something to worry about.
- If you think your brain is working quite well compared to the relevant alternatives, then most changes are bad.
- It is possible to know enough to hurt yourself, but not enough to keep yourself from doing it. Sometimes being incapable of taking your beliefs seriously is an adaptive protection mechanism.
- Humans do experience 'drift'. In general, influences can accumulate. If you're going to start self influencing, it's likely that you'll be successful at doing something, so make sure to do it right.
Arguing against self modifying, you might say that people have spent a long time evolving to do something close to what we want in an environment kinda close to ours now, and that explicit reasoning is more of the public relations guy that we don't and actually shouldn't take seriously. People say "If I could just take a pill and do X, I would" for all sorts of X, and sometimes they actually believe it. If we gave them the power to actually do it, then all this signaling just got dangerous.
But I don't really buy into this story - not completely. For one, evolution is stupid. The people it made aren't strategic - about anything. No automatic strategy? No one behind the wheel? Jesus Christ! Someone fucking drive!
There's an interesting argument that states that we are in the minimally intelligent civilization that has managed to advance this far. After all, intelligence evolves much more slowly than civilization grows, so of course civilization will progress to that point not long after the threshold is reached. This has a ton of fascinating implications that I won't get into (and I'm taking shortcuts), but a species that has just barely learned to use their pre-frontal cortices well enough to accumulate knowledge shouldn't be expected to be doing an optimal job of anything, even given their hardware.
And we know that people learn the wrong things sometimes. Spending a few minutes on youtube can confirm that these are not all adaptive fears. And we shouldn't expect the salient combination of cues to always be something worth being afraid of.
So of course, there really are dangers. Don't give the PR guy root access without consulting everyone else first. That would be stupid. Since the possibility for gain is so large and we're not really safe anyway, we can't just ignore the problem.
So what would it even mean to decide to "not self modify"? Every time two stimuli occur close in time, they get conditioned together whether we like it or not, so we can't lock ourselves in place even if we wanted to. And surely we don't want to completely freeze our connection weight matrix anyway, since that would prevent any learning at all. So we don't have an easy and clean 'null option', but maybe we have a harder and messier one?
One candidate I can think of for a null option is to stay out of the meta levels as much as possible and just do stuff. But that pretty much rules out correcting biases and becoming less wrong, so screw that. Or maybe play on the lower meta levels as much as you feel like, but stay away from the tree of knowledge when it comes to self modification. That doesn't sound very safe either, as people run themselves into the ground all the time, which is why CBT is so helpful.
There's just no nice clean "null" option - at least not one that I can see. People have been using their whole brains all along - there's no magical divide that we shouldn't cross. CBT isn't just some magic that Science has conjured, its also more or less the same as what mentally healthy people do by default. Certainly the first time I heard of CBT, my response was "Duh. I do that. I just never needed to name it"
But I don't really buy into this story - not completely. For one, evolution is stupid. The people it made aren't strategic - about anything. No automatic strategy? No one behind the wheel? Jesus Christ! Someone fucking drive!
There's an interesting argument that states that we are in the minimally intelligent civilization that has managed to advance this far. After all, intelligence evolves much more slowly than civilization grows, so of course civilization will progress to that point not long after the threshold is reached. This has a ton of fascinating implications that I won't get into (and I'm taking shortcuts), but a species that has just barely learned to use their pre-frontal cortices well enough to accumulate knowledge shouldn't be expected to be doing an optimal job of anything, even given their hardware.
And we know that people learn the wrong things sometimes. Spending a few minutes on youtube can confirm that these are not all adaptive fears. And we shouldn't expect the salient combination of cues to always be something worth being afraid of.
So of course, there really are dangers. Don't give the PR guy root access without consulting everyone else first. That would be stupid. Since the possibility for gain is so large and we're not really safe anyway, we can't just ignore the problem.
So what would it even mean to decide to "not self modify"? Every time two stimuli occur close in time, they get conditioned together whether we like it or not, so we can't lock ourselves in place even if we wanted to. And surely we don't want to completely freeze our connection weight matrix anyway, since that would prevent any learning at all. So we don't have an easy and clean 'null option', but maybe we have a harder and messier one?
One candidate I can think of for a null option is to stay out of the meta levels as much as possible and just do stuff. But that pretty much rules out correcting biases and becoming less wrong, so screw that. Or maybe play on the lower meta levels as much as you feel like, but stay away from the tree of knowledge when it comes to self modification. That doesn't sound very safe either, as people run themselves into the ground all the time, which is why CBT is so helpful.
There's just no nice clean "null" option - at least not one that I can see. People have been using their whole brains all along - there's no magical divide that we shouldn't cross. CBT isn't just some magic that Science has conjured, its also more or less the same as what mentally healthy people do by default. Certainly the first time I heard of CBT, my response was "Duh. I do that. I just never needed to name it"
We shouldn't really expect everyone to learn how to control themselves on their own. I mean, babies are fucking stupid, and it looks like we have to learn even basic things on our own. It doesn't surprise me that not everyone learns to do more complicated things like this. And just as we shouldn't expect everyone to figure out CBT on their own, we shouldn't expect anyone to do an even better job without putting some real thought into it and studying.
No comments:
Post a Comment