What changed

Standard Bank Group, in partnership with AWS, has implemented an automated AI risk assessment tool that evaluates generative AI use cases against international AI standards and internal policies. The tool generates reports and dashboards that detail existing controls, effective practices, and areas needing mitigation.

What this means in practice

The assessment runs entirely within Standard Bank’s AWS account, using Amazon S3 for storing documents, code, and test results. AWS Well-Architected Tool handles intake, AWS Step Functions orchestrates the pipeline, and a foundation model on Amazon Bedrock evaluates evidence using a technique called LLM-as-a-judge. This approach allows the bank to maintain control over model access via AWS IAM and ensures that no prompts or outputs are shared with model providers.

The assessment follows four steps: evidence collection, artifact examination and testing, evaluation against controls, and result summarization. A pre-assessment agent on AWS Fargate scans code repositories and documentation, mapping relevant files to predefined questions about risk events. Delivery teams review and correct the pre-populated evidence before finalizing the assessment.

Limitations and availability

The assessment uses a foundation model on Amazon Bedrock without fine-tuning or customization. Model availability varies by AWS Region, so users must check supported models before deployment. The tool is currently in use by Standard Bank Group and is designed to integrate with existing approval processes, with risk committee decisions still based on consistent evidence collected across use cases. The assessment covers eight AWS responsible AI dimensions, including veracity, privacy, transparency, fairness, safety, controllability, governance, and explainability.

Source facts for Standard Bank automates AI risk assessment for generative AI: Proving, one use case at a time, that AI systems follow it is another. Every new generative AI application adds one more system to prove. Teams can now build one in days, but each still needs a risk committee’s approval before it reaches customers or employees.
Original AINewsTrend explainer based on the cited source. This graphic summarizes source statements; it is not an independent benchmark.

Sources and further reading

  1. awsml: How Standard Bank Group automates AI risk assessment for generative AI