In 2020, the Human Values team at BBC R&D North Lab and I won some funding from NESTA to turn their Human Values work into practical tools for policy-making. Building on my experience shepherding the London Internet of Things community into developing the Better IoT assessment tool, I helped them develop a questionnaire for people building digital tools.
The project (and the work done by the team since then) has been shortlisted for a 2023 IBC Social Impact Awards so I thought it would be nice to publish the questionnaire in its entirety with their permission.
The questionnaire interrogates not only someone’s user experience design decisions but the way they run their digital business. Most of the questions are Y/N type questions but some are more suited for exploratory group discussions. The Human Values were defined by Leanne Kerlin and the other team members included Suzanne Clarke, Ian Forrester, Jasmine Cox, Michael Evans and Bill Thompson.
Achieving goals & Receiving recognition
This section is all about making sure people using your service are able to achieve their goals by supporting them, motivating them and removing distractions. It’s also about how you provide external validation that boosts a person’s self-esteem
- Do you use positively phrased language to support people to achieve their goals?
- Do you have processes in place to help when people get stuck?
- Do you mitigate against unnecessary distractions?
- Do you enable people to have manual control of notification tools?
- Do you design for different levels of engagement for the service or product? (e.g. active, passive, superuser)
- Do you personalise your service based on the persons’ income, ethnicity, sexual preference, gender or any other protected characteristic (e.g. by the Charter of Human Rights?) Why?
- Is it clear when recognition/praise is user or service generated?
Workshop questions:
- Do you have metrics based on completion of a task? Why? How is success framed?
- Do you have ways and means of proving that people are acheiving their goals?
- Do you differentiate between your goal as a business and their goal as a person?
- Does your service work more effectively the more recognition/praise people get? How?
- How do people receive recognition on your service?
Being inspired and Exploring the world.
This section is about the means people are given to explore, discover and their ability to express themselves with the product or services you build.
- Have the last 3 advertising campaigns you have produced used people from a range of ethnic backgrounds? (Using services like: Black Illustrations , Cut out Mix, Cut-outs)
- Do you self-report or publicly disclose any digital manipulation?
- Do you discuss your algorithmic procedures with your team?
- Do you offer mechanisms for discovery? (eg. recommendations, curated content)
- Do you offer mechanisms for exploration? (eg. search bar)
- Is content discovery an automated process?
- Do you disclose your algorithmic procedures publicly?
- Do you use plain language to describe those algorithmic procedures. (eg. Crystal Mark)
Workshop questions:
- Do you work with organisations that specialise in diverse imagery? Which?
- Do you have ways to collect information about people taking posiitve action? e.g. stories, do you collect stories, feedback and anecodotes?
- Do you take measures to remove false or highly fabricated information?
- Do you inspire people to take positive action?
- Do you ensure your audience have exposure to a wide range of information?
- How crucial is discoverability to your service?
Being Safe and well
This section focuses on assessing whether you protect people’s safety and wellbeing.
- Can people with cognitive impairments receive important information about your service in a variety of formats? (screen reader, braille, captions, transcripts)
- Do you have processes to assess whether your information is transparent to people?
- Do you have parental control tools?
- Do you maintain a list of offensive language and culturally inappropriate content? Do you have procedures in place to address the use of anything on this list by staff or customers?
- Do you have procedures in place to address the use of anything on this list by staff or customers?
- Do you have a default setting to auto-renew a subscription/after a trial period?
- Do you provide aftercare for customers? In what format?
- Do you force action from a person in a time-based manner (e.g. using countdown)?
- Do you require a person’s consent for any additional costs?
- Do you have ways to mitigate against customers going into debt to use your product?
- Can a person turn off all your notifications with one action?
- Do you make default notifications settings clear?
- Do you explicitly build mechanisms to keep a person on your platform for longer to serve your financial objectives? (e.g autoplay)
Workshop questions:
- Are the T&Cs clear and understandable?
- Do you have security first data strategies?
- Do you have processes to protect the safety of your users?
- If your service is a digital device, is it marked by the appropriate regulator?
- Is your service compliant with latest web accessibility standard? (eg. WCAG) and which one (s) do you follow?
- Do you build in the ability for breaks and time outs?
- Do you ensure you are not increasing distress / Do you have processes to minimise stress responses?
- Do you trigger user anxiety? (e.g. panic buying)
- Is user support provided for your product/service? How is it communicated?
- Describe ways in which you might be misleading users. (e.g. within information, availability)
Belonging to a group and connecting with others
This section is about how people have the means and ability to build a sense of community while using your service or product.
- Do you use gender coded language? (eg. http://gender-decoder.katmatfield.com/about)
- Do you price your products differently according to the gender of a person?
- Do you support the creation of employee groups or networks inside your organisation?
- Do you have processes in place that allow employees to report unacceptable behaviour? (e.g. bullying, hate speech)
- How often do you test your product with people from a variety of ethnicities?
- Do you help people identify groups or communities they could join as part of your product/service?
- If you offer the ability for people to join communities, do they have access to moderation tools?
- Do you publish an annual modern slavery statement?
- Has anyone in your organisation ever been taken court for breaches of the Sexual Offences Act 2003?
- Can people using your service clearly identify an unverified account?
- Can people discover others who also use your product/service?
- Do you help people manage what information they share with others by default?
- Do you have methods to help people find others with shared interests?
Workshop questions:
- Do you publicly report disclose your organisational ethnic diversity?
- Are community-based activities underpinned by popularity metrics? (e.g likes) How do they function?
- Do you help people to manage their own profile’s discoverability ?
- Do you verify user identities? Which mechanisms do you use?
Feeling impactful & Growing myself
This section is about the ability for people to contribute and have an impact in the world as well as the learning and development opportunities that emerge from using a product or service.
- Do you have processes in place that enable your employees to support wider initiatives going on in society?
- Do you allow people to share their level of expertise with others outside the product/service?
- Can your employees develop skills that could be formally recognised/accredited elsewhere?
- Do you provide opportunities for active learning (save for later, bookmarking, tutorials)?
- Do you restrict people from increasing their level of expertise? eg. through financial or other barriers
- Do you allow people to share their level of expertise with others using the product/service?
- Do you make editorial choices to encourage progression in the development of skills?
Workshop questions:
- Do you make editorial decisions to define and encourage contribution to your product/service?
- Have you considered how users might see that their contribution is worthwhile?
- Do you provide support, assurance or guidance for your employees?
- Do you provide support, assurance or guidance as users develop skills?
Having autonomy & stability
This section is about assessing the impact of your product/service on people’s sense of autonomy, how they feel secure and how they might build trust and assurance.
- Do you have key performance indicators associated with how customer service is handled by your organisation?
- Do you regularly review your product experience?
- Do you have a data officer in place to handle GDPR related issues?
- Do you give people control over the way the service represents them? (e.g configuring their profile)
- Do you highlight any revisions to your content? (e.g. published date, addendums/retractions)
- Are people able to delete their profile and personal data from your service?
- Do you highlight in plain language how people’s data is being used by the product or service?
- Do you offer ways to limit usage of your product based on criteria a person can set (for eg. time or time of the day?)
- Do you allow people to opt-out of your direct marketing campaigns based on their collected data?
- Do you stress-test your systems? (e.g. load balancing, ddos)
- Do you provide guidance for using the product or service? (e.g. FAQs, help/about section)
- Do you offer 24h/7 support for people using your product/service?
- Do you offer 24h/7 support for employees?
- Do you offer alternative communication channels in the event of service downtime?
- Do you warn people using your service of any future planned maintenance?
Workshop questions:
- How easily can someone leave your service? Describe this process, for eg. do you require talking to a customer support agent for someone to leave your service?
Pursuing pleasure
- Do you review the way that pleasureable and non-pleasureable activities are designed into your service?
- Do you monetise the enhanced experience/higher tier of your service?
- Do you actively punish people who are choosing not to pay for your service? (e.g. reduced quality)
Workshop questions:
- What percentage of your users pay for your service?
- Do you allow people to share their satisfaction or disatisfaction with your service/product? How?
Understanding & expressing myself
This section is about how your product helps a person gain a clear sense of who they are or helps them express their identity through personality, attitudes and behaviour.
- Do you offer private or incognito mode of accessing/exploring your service?
- Do you show how your service is being used over time by a person? (e.g history or patterns of use)
- Do you have metrics on how customer/user feedback is processed by your team?
- Do you have processes in place to capture employee feedback? (eg. whistleblowing hotline)
- Do you provide personalisation options for people using your service/product?
- Do you have mechanisms for people to leave feedback?
If you have any feedback about this work, reach out to Bill Thompson and Suzanne Clarke.