AF - Consent across power differentials by Ramana Kumar

The Nonlinear Library: Alignment Forum

Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Consent across power differentials, published by Ramana Kumar on July 9, 2024 on The AI Alignment Forum. I'd like to put forward another description of a basic issue that's been around for a while. I don't know if there's been significant progress on a solution, and would be happy to pointed to any such progress. I've opted to go for a relatively rough and quick post that doesn't dive too hard into the details, to avoid losing the thought at all. I may be up for exploring details further in comments or follow-ups. The Question: How do you respect the wishes (or preferences) of a subject over whom you have a lot of control? The core problem: any indicator/requirement/metric about respecting their wishes is one you can manipulate (even inadvertently). For example, think about trying to respect the preferences of the child you're babysitting when you simply know from experience what they will notice, how they will feel, what they will say they want, and what they will do, when you put them in one environment versus another (where the environment could be as small as what you present to them in your behaviour). Is there any way to provide them a way to meaningfully choose what happens? We could think about this in a one-shot case where there's a round of information gathering and coming to agreement on terms, and then an action is taken. But I think this is a simplification too far, since a lot of what goes into respecting the subject/beneficiary is giving them space for recourse, space to change their mind, space to realise things that were not apparent with the resources for anticipation they had available during the first phase. So let's focus more on the case where there's an ongoing situation where one entity has a lot of power over another but nevertheless wants to secure their consent for whatever actually happens, in a meaningful sense. Lots of cases where this happens in real life, mostly where the powerful entity has a lot of their own agenda and doesn't care a huge amount about the subject (they may care a lot, but maybe not as much as they do about their other goals): rape (the perhaps central example invoked by "consent") advertising representative democracy colonisation ("civilising" as doing what's good for them) Our intuitions may be mostly shaped by that kind of situation, where there's a strong need to defend against self-interest, corruption, or intention to gain and abuse power. But I think there's a hard core of a problem left even if we remove the malicious or somewhat ill-intentioned features from the powerful entity. So let's focus: what does it mean to fully commit to respecting someone's autonomy, as a matter of genuine love or a strong sense of morality or something along those lines, even when you have a huge amount of power over them. What forms power can take: brute force, resources that give you physical power support from others (that make you - your interests - a larger entity) intelligence: the ability to predict and strategise in more detail, over longer time horizons, and faster, than the subject you are trying to engage with speed - kinda the same as intelligence, but maybe worth pulling out as its own thing knowledge, experience - similar to intelligence. but maybe in this case emphasising access to private relevant information. Think also of information asymmetry in negotiation. Examples where this shows up in real life already (and where people seem to mostly suck at it, maybe due to not even trying, but there are some attempts to take it seriously: see work by Donaldson and Kymlicka): adaptive preferences children animals (pets, domesticated, and otherwise) disabled people, esp. with cognitive disabilities oppressed/minoritised people and peoples future generations and other non-existent peoples It may be that the only true soluti...