top of page

Apple’s Sensitive Content Warning: Toward Protecting Intimate Privacy on Messaging Platforms

Digital messaging platforms serve as critical spaces for intimate connection, where users yearn to feel that the content they share is completely private. I have previously discussed and empirically demonstrated the significance of the screenshot feature in threatening these digital spaces and ultimately, the trust that can be formed within them. Beyond the screenshot feature, however, messaging platforms face additional challenges that implicate user privacy and safety. For example, cyberflashing. Cyberflashing involves the receipt of unsolicited nude images via a digital messaging platform, resulting in harassment and distress for recipients. In response to this issue, following the precedent set by cyberflashing laws in Texas and California, Apple recently launched its Sensitive Content Warning. Although this new feature has some limitations, it represents a fantastic advance to intimate privacy on messaging platforms. 


So, how exactly does the Sensitive Content Warning work? 


Users with the most recent iOS update can now opt-in to iMessage blurring received images that may contain nudity before it can be viewed. Apple is able to do this through on-device machine learning that allows them to recognize nude images. In contrast to other machine learning models that risk breach and bias, on-device machine learning makes it so that none of the data used for training or analysis leaves the user’s device. Their website notes, “Because they're analyzed on your device, Apple doesn't receive an indication that nudity was detected and does not get access to the photos or videos as a result.” When users click to open the blurred image, iMessage will provide a warning about its potentially nude content. If the recipient wants to view the photo, they are able to do so. If not, this new feature makes it so that they never have to. Although Apple is certainly on the right track toward protecting user privacy and safety within iMessage, their approach just scratches the surface of its potential.


First, Apple did not widely publicize this new feature. When its users updated their software to the new iOS , they were not given notice about the new Sensitive Content Warning. It is highly possible that many Apple users still do not know of this features’ existence. In order for privacy and safety features to be effective, users must be acutely aware of their options to protect themselves.


A related issue here is that the Sensitive Content Warning requires users to opt-in. Presenting privacy features as opt-in have long been discussed as problematic, with many arguing (including myself) that such features should be offered by default. Apple designers, and others who may want to create a similar feature, should ask themselves if there are circumstances where their users would not want this privacy-protective feature. My take is that this feature would cause few, if any, negative consequences. The Sensitive Content Warning should be turned on by default not only to protect receivers from becoming victims of cyberflashing, but also to protect the intimate privacy of the sender. 


Let me explain…


Imagine a situation where someone receives a (potentially welcomed) intimate image from their romantic partner while at dinner with friends. If enabled, Apple’s Sensitive Content Warning would prevent the receiver from unintentionally exposing the image to others who may be in an eye's view of their mobile device. While the marketing of this new feature is geared toward preventing cyberflashing, Apple should recognize that it can have an even broader impact on intimate privacy protection.  


Critically, Apple’s Sensitive Content Warning highlights the capacity for Apple, and other operating systems, to embed obscurity in its messaging platforms through making it more difficult to access intimate content. In my recent dissertation work, I empirically demonstrated that blurring received messages prevented users from collecting screenshots of others’ private messages. I found that operationalizing obscurity in this way—similar to what Apple has done with its Sensitive Content Warning—could help to adjust the existing norms around screenshot collection and sharing. While Apple is currently obscuring access to nude imagery, they should consider how they may extend this feature to mitigate its further dissemination. Protecting against screenshot collection and sharing of intimate imagery is a great place to start in the fight to create messaging platforms that can be truly private and confidential. 


Apple’s Sensitive Content Warning shows promise for technology design that can protect intimate privacy within messaging platforms. It showcases how obscurity of confidential content could work in practice, providing precedent to expand this technology into contexts outside of nude imagery. That being said, there is room for the company to improve the adoption of this feature both through its promotion and implementation.

bottom of page