Director of Research, Institute for Experiential AI at Northeastern University. Co-author of Modern Inf. Retrieval. ACM & IEEE Fellow.
If AI technology and machine learning (ML) were parents, deep learning (DL) would be their most successful child, the one who learns from its own mistakes and makes better decisions than the neural networks of generations before. Although deep learning requires big data and most companies only have access to small data, DL is taking the world by storm. But no parent or child is ever perfect, and as the usage of AI grows exponentially, so have the number of AI incidents.
The most we can ever expect from any parent or child is responsible behavior. At the bare minimum, the same rule should apply to AI technology that impacts billions of people across the world in countless ways. But when the system in question is a black box or, at best, a blurred process, how can we ensure we’re using AI responsibly? How can we minimize the risk of harm to people directly or indirectly?
The answer is simple: You need to have full awareness of what you’re doing, but that only comes by first asking yourself the right questions.
Questions Of Competence
1. Do you have the right experts for the problem that you’re trying to solve?
Many times, data scientists are asked to address issues in unfamiliar disciplines. As important as it is for a data scientist to interpret data, it’s equally important for experts in the field at hand to determine how the data applies to that field. For a complete, well-rounded analysis, you need to have a multidisciplinary team that contests the design and implementation from the beginning to the end.
2. Are you authorized to solve any problems that come up along the way?
Before analysis begins, you should have a full grasp of your limitations. Whatever decision stems from your team’s findings may fall outside the scope of your position because of legal, ethical or even company hierarchy reasons. Having designated team members authorized to address such concerns makes for a more efficient process with all bases covered.
3. Does your team’s solution rely on a proven scientific fact?
Spurious correlations can produce magical results that imply false causality. For example, using facial biometrics to predict sexual orientation is simply a new kind of phrenology, a 19th-century pseudoscience. Make sure you’re able to back up your most impressive or exciting results with tried-and-true science.
4. Is your team’s solution proportional to the problem?
You wouldn’t use a hammer to post a note with a thumbtack. Smaller-scale problems benefit from simple solutions. Deep learning or big data isn’t necessary to solve every problem. Collecting the minimal amount of data per user in the shortest amount of time could provide targeted information without risking errors from going to too large a scale.
5. Is your team’s solution secure, safe and does it comply with all legal regulations?
The best systems are the most trustworthy. Be sure to examine regulations beyond the organizational level. For example, various countries and regions have different policies on data protection. Know all your legal boundaries before you start.
6. Has your team checked for bias in the data? In the objective function of the system? Or in the feedback loop between the users and the system?
Data isn’t the only source of bias. It can occur in the algorithm as well. In the case of Deliveroo, gig-workers who put in fewer hours were judged as unproductive and given lower rankings without accounting for why their hours were lower. The same can be valid for a usage feedback loop that may create a popularity bias or more subtle interaction biases. Remember that replicating the past doesn’t change the future.
7. Is your team using a finite number of categories or arbitrary thresholds that are hard to justify?
Arbitrary thresholds restrict potential outcomes and skew results. The same applies to default/arbitrary thresholds found within ML models. A prime example of this would be the use of race as a category. Race is a social construct, and skin color is a continuous variable! It can’t fit into a predetermined number of labels.
Questions Around Societal Impact
8. Do you have interpretability or explainability in your system?
Interpretable models can tell you how the results are obtained, but often this isn’t enough. Without explainable models or a human in the loop to answer questions when users contest results, there’s little transparency. Depending on your application, explainability may be an essential part of how your company’s system gains trust. But explainable models don’t always work. In the healthcare industry, for example, explainable algorithms may make things worse.
9. Are your users fully aware of the impact of your system on their lives?
Many people still harbor a healthy skepticism when it comes to AI technology. Recent data breaches and ransomware attacks have raised concerns about how personal information is collected and used. Make sure you have explicit consent (or a legal basis) to collect user data. Be transparent about how long you plan to store it and what purpose it serves.
10. Have you deeply considered the direct (and indirect) impact of your product or service?
Here, the accuracy of your model is irrelevant. What matters is the impact of the mistakes you make, even if they are few. In cases where people were falsely accused by facial recognition systems, killed by driverless cars or unethically targeted for fraud, the damage was severe and lasting.
Be Responsible, Ask Good Questions, Make Good Choices
No one expects your systems to be perfect all the time, but you should promote awareness and transparency. Learning is part of the process, especially when it comes to AI. But if you can positively answer these 10 questions, you’re on your way to building a responsible AI system and a product or service people can trust — one that you would gladly use yourself.