DYNAMIC VISUAL TECHNOLOGIES LIMITED

Databricks Specialist Services

Databricks is a cloud-based Unified Data Analytics Platform that uses open source technologies. It combines the latest innovations of the open source community with enterprise-grade functionality. Our services cover the entire delivery lifecycle, including architecture, design, implementation, deployment and support of your Databricks solution.

Features

• Identify platform requirements for data and analytics
• Architect and design Databricks solution
• Implement data pipelines to populate Databricks
• Train users and assist with Databricks adoption
• Provide support packages for Databricks
• Implement ML models and data science in Databricks
• Integrate AI with Power BI in Databricks
• Unify data gathering and model creation in Databricks
• Provision Databricks workspace and cluster
• Use Databricks to train ML models

Benefits

• Ensure fit-for-purpose Databricks solution.
• Use Agile/DevOps for rapid benefits.
• Develop data pipelines for Databricks.
• Use analytics, ML for business benefits.
• Get expert support for peace of mind.
• Databricks support for better decisions.
• Databricks accelerates projects through productivity.
• Extract maximum value from data with Databricks.
• Databricks guidance for best analytic methodologies.
• Reduce effort moving analytics from lab to production.

£350.00 to £950.00 a unit a day

Service documents

• Pricing document

  PDF

• Service definition document

  PDF

• Terms and conditions

  PDF

Request an accessible format

If you use assistive technology (such as a screen reader) and need versions of these documents in a more accessible format, email the supplier at jbramsden@dvt.co.uk. Tell them what format you need. It will help if you say what assistive technology you use.

Framework

G-Cloud 14

Service ID

296015633568121

Contact

Jason Bramsden
020 3422 3400
jbramsden@dvt.co.uk

Service scope

• Software add-on or extension: No
• Cloud deployment model:
  • Public cloud
  • Private cloud
  • Hybrid cloud
• Service constraints: Services outlined in this section are delivered through cloud platforms like AWS and Azure, or via hybrid solutions such as Databricks. DVT imposes no service constraints on the products described here, aside from the limitations specified in the Service Level Agreements (SLAs) and the software or hardware requirements of each provider.
• System requirements: Databricks tool licensing is governed by the respective provider's terms

User support

• Email or online ticketing support: Email or online ticketing
• Support response times: We offer SLA- and KPI-backed support solutions for cloud hosting and software services. Our approach combines agile and traditional practices to meet clients' needs optimally.
• User can manage status and priority of support tickets: Yes
• Online ticketing support accessibility: WCAG 2.1 AAA
• Phone support: No
• Web chat support: No
• Onsite support: Yes, at extra cost
• Support levels: Basic Support: Provides essential assistance during business hours. Includes access to documentation and community forums. Ideal for clients seeking self-help resources or general inquiries. Standard Support: Faster response times and extended coverage (often 24/7). Includes phone and email support. Helps with critical incidents and bug fixes. Premium Support: Dedicated technical account managers (TAMs) provide personalized assistance. Proactive monitoring and priority handling for critical issues. Suited for large enterprises or mission-critical projects. Enterprise Support: Custom SLAs, on-site assistance, and strategic guidance. Direct access to senior engineers and architects. Tailored for Fortune 500 companies or unique project requirements.
• Support available to third parties: Yes

Onboarding and offboarding

• Getting started: Online training, and user documentation
• Service documentation: Yes
• Documentation formats:
  • HTML
  • PDF
  • Other
• Other documentation formats: Markdown file format
• End-of-contract data extraction: Once a subscription with databricks provider ends, typically there is a set window in which users can extract their data. The exact terms will be specified by each client.
• End-of-contract process: Cloud Provider End-of Contract: Access Termination: When a cloud provider contract ends, access to services may cease. Data Retrieval: Data must be retrieved from the cloud environment. Renewal or Transition: Negotiate renewal or explore other providers. DVT's End-of Contract: Service Handover: Ensure a smooth transition of services, and relevant documentation. Knowledge Retention: Retain critical knowledge gained during the engagement. Contractual Obligations: Settle outstanding payments or fees.

Using the service

• Web browser interface: Yes
• Supported browsers:
  • Microsoft Edge
  • Firefox
  • Chrome
  • Safari
• Application to install: No
• Designed for use on mobile devices: Yes
• Differences between the mobile and desktop service: There are no differences, as everything is designed to be cross platform.
• Service interface: No
• User support accessibility: WCAG 2.1 AAA
• API: Yes
• What users can and can't do using the API: Subject to the providers licensing and Api requirements
• API documentation: Yes
• API documentation formats: Open API (also known as Swagger)
• API sandbox or test environment: No
• Customisation available: Yes
• Description of customisation: Tailored to the customer's requirements within the limitations of the service provider

Scaling

• Independence of resources: To ensure a positive user experience, cloud services implement strategies such as scalability, load balancing, and resource isolation. Scalability allows services to handle increased demand without performance degradation. Load balancing distributes requests across servers, preventing overload. Resource isolation allocates separate pools for different users, avoiding interference. Rate limiting and quotas prevent abuse, while caching and CDNs optimize content delivery. Clear SLAs and service guarantees maintain reliability, and monitoring with auto-scaling adapts to changing demands

Analytics

• Service usage metrics: Yes
• Metrics types: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/analyze-metrics
• Reporting types: Real-time dashboards

Resellers

• Supplier type: Reseller providing extra features and support
• Organisation whose services are being resold: Microsoft, AWS, Databricks

Staff security

• Staff security clearance: Conforms to BS7858:2019
• Government security clearance: Up to Baseline Personnel Security Standard (BPSS)

Asset protection

• Knowledge of data storage and processing locations: Yes
• Data storage and processing locations: United Kingdom
• User control over data storage and processing locations: Yes
• Datacentre security standards: Complies with a recognised standard (for example CSA CCM version 3.0)
• Penetration testing frequency: Less than once a year
• Penetration testing approach: Another external penetration testing organisation
• Protecting data at rest:
  • Physical access control, complying with CSA CCM v3.0
  • Physical access control, complying with SSAE-16 / ISAE 3402
• Data sanitisation process: Yes
• Data sanitisation type:
  • Explicit overwriting of storage before reallocation
  • Deleted data can’t be directly accessed
• Equipment disposal approach: Complying with a recognised standard, for example CSA CCM v.30, CAS (Sanitisation) or ISO/IEC 27001

Data importing and exporting

• Data export approach: Users with the relevant permissions: When exporting data from cloud providers like Azure and AWS, users can use services like Azure Import/Export or AWS Data Exports. These services allow secure data transfer by shipping physical disk drives to the cloud provider’s datacenter. It’s like mailing your data for upload, especially useful when internet transfer is slow or costly
• Data export formats:
  • CSV
  • Other
• Other data export formats:
  • Avro
  • Binary
  • Excel
  • JSON
• Data import formats:
  • CSV
  • Other
• Other data import formats:
  • Avro
  • Binary
  • Excel
  • JSON
  • ORC
  • Parquet
  • XML

Data-in-transit protection

• Data protection between buyer and supplier networks:
  • Private network or public sector network
  • TLS (version 1.2 or above)
  • IPsec or TLS VPN gateway
  • Other
• Other protection between networks: To protect data between the buyer’s network and the service provider’s network, robust security measures are essential. These include encryption for secure data transmission, authentication to verify identities, and firewalls to filter traffic. Regular audits and monitoring help detect vulnerabilities. By implementing these safeguards, both parties can ensure data privacy and prevent unauthorized access during communication.
• Data protection within supplier network:
  • TLS (version 1.2 or above)
  • IPsec or TLS VPN gateway
  • Other
• Other protection within supplier network: To protect data within a network, organizations employ robust security measures. These include end-to-end encryption for secure data transmission, firewalls to filter traffic, and network segmentation to isolate critical zones. Regular monitoring, access controls, and employee training enhance security. By combining technical controls and user awareness, data remains confidential and safeguarded within the network.

Availability and resilience

• Guaranteed availability: Tailored to the customers requirement. When designing a high availability databricks solution on both Azure and AWS, consider the following key aspects: Azure Databricks ensures high availability within a single region. Key components include control plane services running in Kubernetes clusters, workspace data stored in replicated databases, and automatic VM handling in specific availability zones. Interregional disaster recovery patterns are also provided.; ; Databricks on AWS leverages Delta Lake for ACID guarantees. It uses transactional commits for writes to cloud object storage, ensuring data consistency and reliability.
• Approach to resilience: Tailored to the customers requirement. Azure Databricks prioritizes reliability through the following design elements:; ; Delta Lake: Leveraging Delta Lake, it provides ACID transactions, schema enforcement, and unified streaming/batch processing.; Apache Spark or Photon: Internal Spark task failures are automatically rescheduled, ensuring job execution continuity.; Rescued Data Handling: Invalid data is filtered out during ingestion, and tools like Auto Loader and Delta Live Tables handle rescued data.; Databricks on AWS shares similar principles:; ; Delta Lake: It also uses Delta Lake for reliability, offering ACID transactions and schema enforcement.; Apache Spark or Photon: Automatic rescheduling of failed Spark tasks enhances resilience.; Rescued Data Handling: Similar to Azure Databricks, it filters out invalid data during ingestion using tools like Auto Loader and Delta Live Tables.
• Outage reporting: 1. Azure: Azure Status Page: Azure maintains a dedicated status page that provides real-time information about service health and any ongoing incidents. Users can visit this page to check the current status of Azure services, including any reported outages or disruptions. Azure Service Health: Azure Service Health is a comprehensive dashboard that displays the health of Azure services in your specific region. It includes information about incidents, planned maintenance, and historical data. Application Insights: If you’re using Azure Application Insights for monitoring your applications, you can access the SLA workbook template within Application Insights. 2. AWS: AWS Status Page: AWS maintains a similar status page where users can check the current status of AWS services. It provides real-time updates on incidents, outages, and planned maintenance. Third-Party Monitoring Tools: Websites like Downdetector track user-reported issues and outages for various cloud providers, including AWS. Service Health Dashboard: Within the AWS Management Console, you can access the Service Health Dashboard. It provides information about the operational status of AWS services in different regions.

Identity and authentication

• User authentication needed: Yes
• User authentication:
  • 2-factor authentication
  • Public key authentication (including by TLS client certificate)
  • Identity federation with existing provider (for example Google Apps)
  • Limited access network (for example PSN)
  • Dedicated link (for example VPN)
  • Username or password
  • Other
• Other user authentication: Authentication is the process of verifying the identity of a user or system before granting access to a service. Common methods include username and password, OAuth2 and OpenID Connect (OIDC) for single sign-on, API keys for programmatic access, biometrics (like retina scans or fingerprints), and certificates (such as TLS certificates for secure communication). Remember that authorization complements authentication by defining access policies for specific resources within the service.
• Access restrictions in management interfaces and support channels: Azure Databricks: Prioritizes reliability through ACLs, Delta Lake, and automatic data rescue. Management interfaces are secured via Azure AD authentication.; ; Databricks on AWS: Similar principles apply, with IAM managing access control. Both platforms ensure secure management interfaces and support channels
• Access restriction testing frequency: Less than once a year
• Management access authentication:
  • 2-factor authentication
  • Public key authentication (including by TLS client certificate)
  • Username or password

Audit information for users

• Access to user activity audit information: Users contact the support team to get audit information
• How long user audit data is stored for: At least 12 months
• Access to supplier activity audit information: Users contact the support team to get audit information
• How long supplier audit data is stored for: At least 12 months
• How long system logs are stored for: Between 1 month and 6 months

Standards and certifications

• ISO/IEC 27001 certification: Yes
• Who accredited the ISO/IEC 27001: QAS International
• ISO/IEC 27001 accreditation date: 27/06/2022
• What the ISO/IEC 27001 doesn’t cover: Control 14.2.7
• ISO 28000:2007 certification: No
• CSA STAR certification: No
• PCI certification: No
• Cyber essentials: Yes
• Cyber essentials plus: No
• Other security certifications: Yes
• Any other security certifications: ISTQB

Security governance

• Named board-level person responsible for service security: Yes
• Security governance certified: Yes
• Security governance standards: ISO/IEC 27001
• Information security policies and processes: DVT adheres to stringent Information Security policies and processes that safeguard our data and operations. ; We are certified with ISO/IEC 27001: 2013 and are currently working towards implementing the ISO/ IEC 27001:2022 standard. ; We are also certified with Cyber Essentials, reflecting our commitment to robust security practices. ; ; We maintain our Information Management Security System (SMS) with regular reviews, updates, and improvements to address security threats and changes within our environment. These are reviewed monthly at CAB (Change Advisory Board). ; ; Our Information Security Office, who is also our MD, is responsible for overseeing the implementation and management if our policies and processes. Our MD reports to our CEO.; ; Training and awareness is done in phases at regular intervals, ensuring all employees are aware of all information security policies and processes in place, helping them understand their role and responsibilities within the ISMS. The ISMS is monitored and reviewed on a continuous basis, with internal audits performed by our infosec team, as well as external audits performed annually by an external body to ensure compliance. We have incident reporting in place, to handle any security breaches or policy violations, with logs and reports produced for review at CAB and by our auditors.

Operational security

• Configuration and change management standard: Supplier-defined controls
• Configuration and change management approach: In Agile and Scrum methodologies, configuration management involves version control, artifact repositories, and Infrastructure as Code (IaC). Change management includes user stories, backlog refinement, sprint planning, daily standups, sprint reviews, and retrospectives. For deployment on Azure and AWS, use Azure DevOps or AWS CodePipeline for CI/CD. Azure offers Azure Resource Manager (ARM) and App Service, while AWS provides CloudFormation, Elastic Beanstalk, ECS/EKS, and Lambda. Prioritize automated testing, monitoring, and rollback strategies for resilience.
• Vulnerability management type: Supplier-defined controls
• Vulnerability management approach: Azure Vulnerability Management: Automated vulnerability scanning tools identify security weaknesses in Azure resources. Regular patch management keeps systems up-to-date. Third-party software patches are also addressed promptly. AWS Vulnerability Management: AWS employs an incident response team and centralizes security tools like Amazon GuardDuty and AWS Security Hub. AWS Config tracks assets, and Amazon Inspector identifies vulnerabilities. Incident response and remediation follow predefined actions. Both Azure and AWS rely on security advisories, threat intelligence feeds, and community resources for information on potential threats. Continuous monitoring and staying informed are crucial for maintaining a secure cloud environment.
• Protective monitoring type: Supplier-defined controls
• Protective monitoring approach: Azure offers comprehensive protective monitoring processes: Azure Monitor collects telemetry from VMs across Azure, AWS, and GCP. Azure Arc-enabled Kubernetes ensures consistent monitoring for AKS and Kubernetes clusters in AWS EKS or GCP GKE instances. Application Insights extends monitoring to applications hosted outside Azure. For AWS: Microsoft Defender for Cloud automates deployment of Azure Arc agents to AWS EC2 and GCP VM instances. File Integrity Monitoring maintains file integrity on VMs. Vulnerability Assessment and OS Hardening secures VMs.
• Incident management type: Supplier-defined controls
• Incident management approach: Azure Incident Management: Pre-defined Processes: Azure recommends creating an incident response guide, prioritization procedures, and testing security response procedures. User Reporting: Users can report incidents through Azure Monitor alerts or directly to the IT team. Incident Reports: Azure services like Microsoft Defender for Cloud and Sentinel automate incident response. AWS Incident Management: Pre-defined Processes: AWS provides an overview of responding to security incidents within a customer’s AWS Cloud environment. User Reporting: Users can report incidents via AWS support or directly to their IT teams. Incident Reports: AWS IQ experts guide customers through security incident responses.

Secure development

• Approach to secure software development best practice: Conforms to a recognised standard, but self-assessed

Public sector networks

• Connection to public sector networks: No

Social Value

• Social Value:

  Social Value

  • Fighting climate change
  • Tackling economic inequality
  • Equal opportunity
  • Wellbeing

  Fighting climate change

  DVT’S commitment to having a positive impact on society is an enduring one. We believe we have a responsibility to our communities, both those we work in and beyond, to help fight climate change.; ; We are determined to become a low carbon business. Our single, virtual London office has achieved the ‘GOOD’ rating under BREEAM, the world’s foremost environmental rating system for buildings. We have made a worldwide, science-based commitment, by decarbonising our operations and supply chain; embedding ESG-related factors across our work; championing non-financial reporting practices, and collaborating with businesses, policymakers and accelerate the transition to net zero.; ; Our adherence to ISO14001 Environmental Management Standards, coupled with our Carbon Reduction Plan commitment to achieve net zero emissions by 2030, is supported by initiatives/policies such as:; •hybrid working model and provision of tools for staff to work from anywhere; creating a sustainable balance between working remotely, in offices and on client sites; •supporting green transport options; cycle to work scheme; •providing locally-based consultants, minimising travel; •reducing paper usage, managing/storing documents electronically and controlling the use of print; •eliminating corporate use of single-used plastics; •using MS teams for company and client meetings; •partnering with local businesses to engage local resources in contract delivery; •promote the benefits of engaging local people and local suppliers through our supply chain; ; Promote environmental volunteering opportunities, our employees can volunteer with environmental charities of their choice.

  Tackling economic inequality

  We believe that we have a responsibility to our communities, both those we work in and beyond to help reduce inequality, raise aspirations and promote social and digital inclusion.; ; We provide long standing support for communities across the UK and South Africa, by sharing resources and skills through volunteering, fundraising and in-kind support. Our ethos is to promote social inclusion and sustainable development by supporting charities and partnering with a social partner/s. ; ; Our robust, diverse local ecosystem is strongly focused on improving local economies and providing access to knowledge/skills for local people/business; meaning less money and skills leave the local community/economy. ; We:; •work in partnership with local businesses, purchasing of resources required to fulfil the contract locally, through SMEs with a registered local footprint.; •promote the benefits of engaging local people and suppliers through our supply chain, challenging negative preconceptions, removing barriers.; •champion SMEs and micro-businesses, in partnership with the local Chambers/Local Enterprise Partnerships/VSCE infrastructures.; •will sign up to Social Enterprise UK’s Buy Social initiative.; •work with our supply chain to raise awareness, knowledge of modern slavery, fair working practices and encourage them to adopt best practice; we’ll include modern salary, fair payment terms and code of ethics in our supplier onboarding process, with regular progress, audit performance updates

  Equal opportunity

  DVT has a core focus on ensuring Diversity, Equality and Inclusion (DE&I) throughout our working practices:; •All colleagues complete DE&I training in respect of internal working practices/services; •Use various recruitment techniques to attract diverse candidates; posting jobs with clear qualifications/skill needs; avoiding language which might dissuade candidates from minority backgrounds; using several CV-assessors to avoid bias.; •Offer skills development/job creation opportunities to attract candidates from diverse backgrounds; apprenticeship schemes, retraining/upskilling initiatives, internal qualifications attainment, employment coaching/mentoring.; •Offer flexible working to accommodate parental responsibilities, disabilities, religious/cultural requirements.; •We report on the gender composition of our workforce and pay is set by role, not gender.; •Our staff demographic: 29% Female and 71% Male. 29% Black, 10% Asian, 6% Mixed and 43% white. 1% with Disabilities. ; ; Our accreditations include: ; We’ve zero tolerance to slavery and human trafficking and employ systems in our business/supply chain to:; •Identify, assess and monitor potential risks of slavery and human trafficking; •Mitigate/resolve identified risk; •Protect whistle-blowers; Our dedicated Delivery Assurance and Support Team ensure all partners/sub-contractors sign up to and comply with our Anti-Slavery & Human Trafficking Policy. We exercise rights of audit to ensure we enforce the policy.

  Wellbeing

  We publish both our long and short term targets for our wellbeing programme, and report on progress in our Annual Report and Non-financial scorecard, which is externally assured.; ; We have a global wellbeing approach which includes the provision of resources and support to improve physical, emotional and mental health and wellbeing.; ; For this service, we will:; ; Deliver wellbeing sessions for Black colleagues, launched as part of our commitment to the Black Lives matter movement.; ; Support the wellbeing of our LGBTQIA+ community UK employee inclusive network for LGBTQ+ people, allies and supporters.; ; Provide workplace adjustments for colleagues who have a disability or long-term condition in relation to all aspects of employment.; ; Design and deliver a comprehensive mental health and wellbeing plan with initiatives that include mental health first aider training, a DE&I outreach programme and facilitating Wellbeing for Women workshops.; ; Provide resources and tools to help our staff develop wellbeing skills, as well as building their resilience and performance. This includes a digital fitness app which is available to all our employees, as well as their family, friends and the wider community.; ; Create an environment that allows open conversations around mental health, align with our mental health advocates network, a group of directors and partners who have experience of mental health problems and who will listen and signpost to the right support.

Pricing

• Price: £350.00 to £950.00 a unit a day
• Discount for educational organisations: No
• Free trial available: No

Service documents

• Pricing document

  PDF

• Service definition document

  PDF

• Terms and conditions

  PDF

Request an accessible format

If you use assistive technology (such as a screen reader) and need versions of these documents in a more accessible format, email the supplier at jbramsden@dvt.co.uk. Tell them what format you need. It will help if you say what assistive technology you use.