Responsible Considerations to Select a Model
Why the choice matters
The story: Hiring the wrong manager affects everything: how customers are treated, how staff feel, and whether the shop makes money. So you interview candidates properly.
In AI/AWS terms: Model selection affects everything from user experience to profitability. Interview your candidates with Model evaluation on Amazon Bedrock or SageMaker Clarify.
For the exam: Evaluate candidate models with Bedrock model evaluation or SageMaker Clarify.
Define the use case narrowly
The story: "A net" isn't a plan. A net for catching a lost pet in the woods should catch everything that moves, even if you get some squirrels, because missing the pet is the worst outcome. A net at an airport security gate that stops dozens of innocent travelers is useless; it has to catch only the right person.
In AI/AWS terms: A narrowly defined use case lets you tune the model for it. Face recognition is a technology, not a use case.
- Gallery retrieval (finding missing persons) favors recall: better to return many possible matches than miss the person.
- Celebrity recognition or virtual proctoring favors precision: too many results are not useful.
The same idea with generative AI in an online store:
| Catalog a product | Persuade to buy | |
|---|---|---|
| Audience | Broad | Narrow |
| Risks | Veracity | Veracity, unwanted bias, toxicity |
| Tuning | Neutral, clear, complete | Focused on what matters most to that group |
For the exam: Missing a match is worse → favor recall. Too many false matches is worse → favor precision.
Performance factors
The story: Comparing candidates for the manager job: how much training they'll need, how experienced they are, whether they come through an agency or you employ them directly, what their contract lets them do, how much they can keep in their head at once, and how fast they respond. And a candidate's score depends on the test you gave them, so if the job changes, retest.
In AI/AWS terms:
- Level of customization: from prompting to full retraining
- Model size: parameter count
- Inference options: self-managed or API
- Licensing: some licenses restrict commercial use
- Context window: how much fits in one prompt
- Latency: time to generate output
Performance is a function of the model and the test dataset, not the model alone. Datasets evolve, so track both.
For the exam: A model's performance depends on the model and the dataset it's tested on.
Other considerations
The story: A responsible business also asks: will this still be good for staff, the town, and the planet in ten years? Does the manager share our values, know when to ask the owner, and own their decisions? How much electricity does the new equipment burn? Does it save money only by laying off half the town?
In AI/AWS terms:
- Sustainability: socially, environmentally, and economically sustainable over the long term.
- Responsible agency: value alignment, responsible reasoning, an appropriate level of autonomy with human oversight, and transparency and accountability.
- Environmental: energy consumption, resource use (GPUs and data centers), and environmental impact assessments.
- Economic: efficiency gains weighed against job displacement, inequality, and concentration of power in a few companies.
For the exam: Responsible model selection also weighs sustainability, responsible agency, environmental impact, and economic impact.