ADVAI LTD

Adversarial Artificial Intelligence Robustness Services For AI Benchmarking and Assurance

A managed service to test and evaluate; cohering Advai's AI Robustness tools for adversarial stress testing, evaluation and red-teaming of neural-network-based AI. Critical in creating assured and robust AI that is resilient to real-world conditions, our platforms align with MLOps, and are suitable for both technical and non-technical users.

Features

• Test and Evaluate AI systems for strengths and weaknesses
• Automated Stress Tests for Computer Vision, OCR AI and NLP
• Large Language Model evaluation and guardrails
• Automated Adversarial AI Red Team stress tests
• Flexible, cross-platform support to consistent design principles.
• Cloud-agnostic, capable of deployment on cloud and on premise.
• Integrated monitoring solutions to track performance, bugs and resource.
• Integrated and automated testing as part of a DevOps pipeline
• Scalable architecture, with easy configurability of CPU, GPU, etc resources.
• Modular, containerised design capable of bespoke configuration

Benefits

• Build AI systems to be secure, safe, and responsible.
• Configure AI to by robust, resilient and accurate
• Stress test AI systems to understand strengths and weaknesses
• Benchmark and select the best model for your use case
• Prioritise MLOps by identifying weaknesses at an earlier stage
• Understand vulnerabilities to Adversarial AI
• Identify, mitigate, and manage risks associated with AI deployment
• Safely implement Large Language Models
• Assure your AI system based on risk, regulation, and policy

£30,000 a licence a month

Service documents

• Pricing document

  PDF

• Service definition document

  PDF

• Terms and conditions

  PDF

Request an accessible format

If you use assistive technology (such as a screen reader) and need versions of these documents in a more accessible format, email the supplier at contact@advai.co.uk. Tell them what format you need. It will help if you say what assistive technology you use.

Framework

G-Cloud 14

Service ID

845462447373893

Contact

David Sully
07572990749
contact@advai.co.uk

Service scope

• Service constraints: Advai pushes releases in line with its scrum cycle every 2 weeks. These are minor release or bug fix releases. Support is limited to the containerised environments that are deployed. Any customisation not agreed with Advai is not supported.
• System requirements:
  • Linux/Windows with WSL/AWS Hosting
  • Docker
  • PostgreSQL
  • Custom domain License for Web Application Firewall
  • Cloud Provider Specific Managed Services
  • MongoDB
  • Enterprise Identity Provider (optional)

User support

• Email or online ticketing support: Yes, at extra cost
• Support response times: Weekdays. 0900-1700; Upto 3 hours
• User can manage status and priority of support tickets: No
• Phone support: Yes
• Phone support availability: 9 to 5 (UK time), Monday to Friday
• Web chat support: No
• Onsite support: Yes, at extra cost
• Support levels: The support requirement will be defined with the Customer during the Discovery phase, and costed according to the Service Delivery Document and Pricing Document.
• Support available to third parties: Yes

Onboarding and offboarding

• Getting started: Advai's capability sits at the cutting edge of AI technology and development. It is often the first time that users have encountered such a capability (there are no known market equivalents for stress testing and red teaming AI). ; ; We therefore advocate a collaborative approach to determine the task requirements (Advai Advance) and define the features of the required service and platform(s). We can conduct onsite training undertaken during the final stages of integration as part of a capability handover. ; ; We can also offer the ability to second personnel for side-by-side training with our specialists.
• Service documentation: Yes
• Documentation formats:
  • HTML
  • ODF
  • PDF
  • Other
• Other documentation formats: Confluence
• End-of-contract data extraction: Our architecture enables the extraction of data held within it in a standard format. Once completed, the virtual storage will be collapsed.
• End-of-contract process: There will be an additional cost associated with the removal of any on-premise architecture or deployment. ; ; On completion of the contract, there will be additional work to remove users, APIs, and any other mechanism put in place specific to that contract. This will be costed and resourced in a similar manner to the contract, and final price will be agreed with the Client clearly and in advance.

Using the service

• Web browser interface: Yes
• Using the web interface: The users can register for an account, which needs to go through an internal approval process, where they can be permissioned and given the relevant access.; ; The functionalities of the web interface will be updated and new features will be added over the course of the framework.
• Web interface accessibility standard: None or don’t know
• How the web interface is accessible: The interface is focused on delivering through a functional design. Features have not been validated for accessibility, but this is in the roadmap.
• Web interface accessibility testing: None has currently been performed. This is in the roadmap and can be accelerated if required.
• API: Yes
• What users can and can't do using the API: The service cannot be setup through the API. Service setup requires execution of Infrastructure as Code to be run before (tasks, facial verification, results and guardrails) can be accessed.; ; Users can push task configurations or requests for task to be executed through the API. Currently this only support tasks for testing. Guardrails only supports LLM queries via a guardrails API.; ; Should a customer require additional API access we would look to provide external endpoints for platform specific task that use currently protected APIs. ; ; Based on the customer requirement we would look to extend, create or provision access to APIs.
• API automation tools: Terraform
• API documentation: Yes
• API documentation formats: Open API (also known as Swagger)
• Command line interface: Yes
• Command line interface compatibility: Linux or Unix
• Using the command line interface: User will need to connect to a deployed instance of our service library.; ; They can invoke tasks from the command line, and pass in a configuration by invoking setup stored in the database or by passing in JSON task configuration. This allows for local files to be used or hosted files to be invoked when a unique key is provided, which is managed by the platform.; ; CLI access does not allow users, to track task, manage datasets and models or user information.

Scaling

• Scaling available: Yes
• Scaling type: Automatic
• Independence of resources: Any UK Gov contract or engagement will be supported by dedicated resource scaled to the requirements of the commitment.
• Usage notifications: Yes
• Usage reporting: Email

Analytics

• Infrastructure or application metrics: Yes
• Metrics types:
  • CPU
  • Disk
  • HTTP request and response status
  • Memory
• Reporting types: Real-time dashboards

Resellers

• Supplier type: Not a reseller

Staff security

• Staff security clearance: Conforms to BS7858:2019
• Government security clearance: Up to Developed Vetting (DV)

Asset protection

• Knowledge of data storage and processing locations: Yes
• Data storage and processing locations: United Kingdom
• User control over data storage and processing locations: Yes
• Datacentre security standards: Managed by a third party
• Penetration testing frequency: At least once a year
• Penetration testing approach: Another external penetration testing organisation
• Protecting data at rest:
  • Physical access control, complying with CSA CCM v3.0
  • Encryption of all physical media
• Data sanitisation process: Yes
• Data sanitisation type: Explicit overwriting of storage before reallocation
• Equipment disposal approach: In-house destruction process

Backup and recovery

• Backup and recovery: Yes
• Backup controls: This is determined in consultation with the end user
• Datacentre setup: Multiple datacentres with disaster recovery
• Scheduling backups: Users contact the support team to schedule backups
• Backup recovery: Users contact the support team

Data-in-transit protection

• Data protection between buyer and supplier networks:
  • Private network or public sector network
  • TLS (version 1.2 or above)
  • IPsec or TLS VPN gateway
• Data protection within supplier network:
  • TLS (version 1.2 or above)
  • IPsec or TLS VPN gateway

Availability and resilience

• Guaranteed availability: Our service is designed with high availability, with redundancy inherent within the architectural design. ; ; We will work with the Client to understand the availability required from the system, and determine how that can be achieved.
• Approach to resilience: Advai's architecture is designed for resilience, which can be enhanced if required. More detail is available on request.
• Outage reporting: Outages will be reported via email alerts to the Client.

Identity and authentication

• User authentication: 2-factor authentication
• Access restrictions in management interfaces and support channels: Segregated and defined user lists will be used to segregate user groups, including management and administrative personnel.
• Access restriction testing frequency: At least once a year
• Management access authentication: 2-factor authentication
• Devices users manage the service through: Directly from any device which may also be used for normal business (for example web browsing or viewing external email)

Audit information for users

• Access to user activity audit information: Users contact the support team to get audit information
• How long user audit data is stored for: At least 12 months
• Access to supplier activity audit information: Users contact the support team to get audit information
• How long supplier audit data is stored for: At least 12 months
• How long system logs are stored for: Between 1 month and 6 months

Standards and certifications

• ISO/IEC 27001 certification: No
• ISO 28000:2007 certification: No
• CSA STAR certification: No
• PCI certification: No
• Cyber essentials: Yes
• Cyber essentials plus: Yes
• Other security certifications: No

Security governance

• Named board-level person responsible for service security: Yes
• Security governance certified: No
• Security governance approach: We are working towards ISO/IEC 27001 ; ; Our internal audit will take place in May 2024. Our external audit and certification will take place between June and August 2024.; ; Security Governance is managed by company policies and controls. The standard procedure for all changes are: - Request for change; - Review and Risk analysis; - Approval for change; - Implement new resource access for defined period or until review; - Review resource access; - Update resources or permissions. All data is encrypted at rest, and appropriate audit controls put in place to track access and changes.
• Information security policies and processes: The company has the following policies:; -IT Security Policies;; -Acceptable use policies;; -Data Loss Prevention;; -Data Breach Policy and Reporting Template.; ; All breaches are reported to IT Security Administrator and follow the steps outlined in each policy.; ; At a high level these steps will have tiers of severity with corresponding SLAs for notifying affected parties. ; - Raise alert of breach;; - Initiate Review and Escalation steps;; - Communicate status to all required parties;; - Initiate Check-in meetings for updates;; - Resolve/Mitigate/Track issue;; - Finish escalation process and notify of closure/tracking;; - Review and learn from event.; ; To promote that policies are followed staff are given initial and annual training.

Operational security

• Configuration and change management standard: Supplier-defined controls
• Configuration and change management approach: Advai uses a feature management process that assesses the requirements of an update or addition before integrating it within our sprint cycle. Product Owners agree on the relative priorities. Any feature is code-checked and security scanned during testing. Any security-critical feature is fast-tracked through the process.
• Vulnerability management type: Supplier-defined controls
• Vulnerability management approach: Our service continually monitors for threats. If a vulnerability is identified we will determine if the rectification belongs with the cloud supplier, Client hosting, or ourselves. In every-case we will determine the threat, risk and issues associated with the vulnerability and the services it impacts. Advai will close down non-critical services before engaging with the Client, actions for critical services will be pre-agreed with the Client. A rectification plan will be created and agreed with the Client, this will determine the speed of any patch deployment.
• Protective monitoring type: Supplier-defined controls
• Protective monitoring approach: Our service continually monitors for compromise. If a compromise is identified we will determine if the rectification belongs with the cloud supplier, Client hosting, or ourselves. In every-case we will determine the threat, risk and issues associated with the compromise and the services it has impacted. ; ; In the event of a compromise Advi will temporarily suspect all user accounts associated with the assessed attack vector and seek to re-verify all identification across the platform. ; ; We will work with the client to adapt to their own protective monitoring approach and integrate any alternative procedures into our own.
• Incident management type: Supplier-defined controls
• Incident management approach: Advai's architecture is designed to respond to a range of incidents, primarily via it's scalable containerised approach. ; ; Our incident management approach is integrated into our feature management plan.

Secure development

• Approach to secure software development best practice: Conforms to a recognised standard, but self-assessed

Separation between users

• Virtualisation technology used to keep applications and users sharing the same infrastructure apart: No

Energy efficiency

• Energy-efficient datacentres: Yes
• Description of energy efficient datacentres: We utilise Amazon Web Services

Social Value

• Social Value:

  Social Value

  • Fighting climate change
  • Tackling economic inequality

  Fighting climate change

  As an AI testing and evaluation company, we recognise the critical role that Artificial Intelligence plays in combating climate change. Identifying risks early in AI projects is paramount; it not only ensures the viability of environmental solutions but also significantly lowers project costs by preventing late-stage failures and expensive corrective measures. Through meticulous data analysis, we help optimise the performance of AI-driven climate initiatives, thereby reducing the costs associated with data management. This optimised data handling enhances the effectiveness of AI applications in monitoring environmental changes and managing renewable resources. Moreover, our expertise in developing robust AI systems increases their deployment success rates, crucial for reliable and sustainable environmental management tools. By ensuring these systems require minimal retraining, we also cut down on computational costs, aligning our technological advancements with sustainability goals. Thus, our role in testing and evaluating AI fortifies the link between advanced technology and effective climate action, ensuring that AI solutions are both impactful and economically feasible.

  Tackling economic inequality

  Our position as a sustainably growing AI testing and evaluation company that recruits from across the UK allows us to bring a broad spectrum of perspectives to our company culture. Our focus on technical credibility during the interview process ensures that we onboard professionals who are not only technically proficient but also ready to contribute to our innovative work culture.

Pricing

• Price: £30,000 a licence a month
• Discount for educational organisations: No
• Free trial available: No

Service documents

• Pricing document

  PDF

• Service definition document

  PDF

• Terms and conditions

  PDF

Request an accessible format

If you use assistive technology (such as a screen reader) and need versions of these documents in a more accessible format, email the supplier at contact@advai.co.uk. Tell them what format you need. It will help if you say what assistive technology you use.