Databricks Data Intelligence Platform

Service scope

Software add-on or extension: Yes
What software services is the service an extension to: Databricks assumes existence of a storage layer in the cloud of choice, S3 in AWS, ADLS in Azure or GCS in Google Cloud. Data is both read and written to the storage services within users cloud account.
Cloud deployment model: Public cloud
Service constraints: Databricks operates ontop of cloud storage layer, necessitating data storage in S3 for AWS, ADLS for Azure, or GCS for Google Cloud. It functions through a fleet of VMs within the user's cloud account. Utilizing processes in Databricks incurs costs, encompassing VM usage duration and storage fees for data retrieval from storage into VMs. The buyers should be mindful of these implications to effectively manage expenses associated with compute and storage usage while leveraging the Databricks platform. Whilst the majority of the cloud's virtual machines are available, not every VM is available to be used with Databricks in every cloud.
System requirements: "Cloud subscription is required "

"Available on AWS, Azure or GCP "

"Data must be stored in S3, ADLS, or GCS "

"Requires a service principal with permissions to create VMs "

"Requires a service principal with permissions to manage storage "

Each workspace requires two subnets

User support

Email or online ticketing support: Yes, at extra cost
Support response times: It depends on the service package purchased, with response times ranging from within 1 business day, to within 1 hour. The most comprehensive package provides an SLA of under one hour response time for production critical workloads, 24*7*365. The minimum response SLA is less than 1 business day. Weekend support is offered for issues with production workloads only.
User can manage status and priority of support tickets: No
Phone support: No
Web chat support: Yes, at an extra cost
Web chat support availability: 9 to 5 (UK time), Monday to Friday
Web chat support accessibility standard: WCAG 2.1 AA or EN 301 549
Web chat accessibility testing: Our platform is designed with accessibility in mind, however we have not conducted specific testing with assistive technology users for our chat support. We continuously strive to improve our services and are committed to enhancing accessibility to meet the diverse needs of all our users.
Onsite support: Yes, at extra cost
Support levels: "Databricks provides tiered support levels to meet the diverse needs of our customers, ensuring optimal platform performance and user satisfaction. Our support structure is designed to offer scalable solutions, from basic problem-solving to advanced technical guidance.

Business Support: Access to our comprehensive knowledge base, community forums, and email support. Suitable for customers with general support needs.
Enhanced Support: Includes all features of Standard Support with the addition of 24/7 phone support and faster response times. This level is ideal for businesses requiring immediate assistance for critical issues.
Production Support: Offers all the benefits of Enhances Support, plus a dedicated Technical Account Manager (TAM). The TAM serves as a proactive and strategic guide, helping to optimize your Databricks deployment and address complex operational challenges.
Additionally, Enterprise customers benefit from the expertise of a Cloud Support Engineer who provides in-depth technical assistance and performance optimization strategies tailored to your specific cloud environment.

Each support level is crafted to ensure that every user, from small enterprises to large corporations, receives the assistance they need to thrive in today's data-driven world."
Support available to third parties: No

Onboarding and offboarding

Getting started: All Databricks customers receive access to self paced learning on the Databricks Academy, which covers areas such as platform administration, data engineering, analytics and data science. In addition to that, there are options such as "train the trainer", instructor led classes, workshops and hackathons.
Service documentation: Yes
Documentation formats: HTML
End-of-contract data extraction: The vast majority of customer data never leaves the customer cloud account. When a customer leaves Databricks, their workspace is deleted within 30 days of workspace or account deletion. Any data contained in the workspace that is not in customer's cloud account can be extracted by the customer at will before the customer leaves.
End-of-contract process: At the end of the committed capacity contract either (a) a renewal may be agreed or (b) the customer's workspace remains accessible but billing reverts to the pay-as-you-go cost model paid through monthly invoices or on a payment card.

Using the service

Web browser interface: Yes
Supported browsers: Microsoft Edge

Firefox

Chrome

Safari

Opera
Application to install: No
Designed for use on mobile devices: No
Service interface: Yes
User support accessibility: WCAG 2.1 AA or EN 301 549
Description of service interface: Databricks' primary interface is through the web UI. This provides an easy-to-use graphical interface to workspace folders and their contained objects, data objects, and computational resources. Interactive development is facilitated through the Databricks notebook, which are similar in concept to Jupyter notebooks, but come with additional features such as automatic versioning, co-presence for collaboration, multi language support, and access controls. There is also a comprehensive API suite for programmatic interactions, which can be accessed by third party tools for orchestrating wider system workflows.
Accessibility standards: WCAG 2.1 AA or EN 301 549
Accessibility testing: "Databricks provides a robust, web-based interface that facilitates efficient and precise data management and analysis. Our integrated workspace supports a broad spectrum of data operations, from data science and engineering to business analytics.

Please note, while Databricks strives to provide an accessible platform, specific testing with assistive technologies has not been conducted. We are committed to continuous improvement in this area to ensure our services are accessible to all users, including those utilizing assistive technologies."
API: Yes
What users can and can't do using the API: A comprehensive set of REST APIs covers all aspects of the platform. For example, managing permissions, managing processing jobs, managing compute resources, managing code repositories. These APIs require authentication through a token generated and stored securely within the Databricks platform. Full API documentation is available on the Databricks website.
API documentation: Yes
API documentation formats: Open API (also known as Swagger)

HTML
API sandbox or test environment: No
Customisation available: No

Scaling

Independence of resources: Each Databricks workspace is isolated in the customer's cloud environment. This means that all demanding operations (for example using compute resource from cloud provider, writing to cloud storage) are independent from what other customers are doing.

Analytics

Service usage metrics: Yes
Metrics types: Within the UI, billable usage metrics are displayed on a per workspace basis. Automatic logging tracks compute resource usage, user auditing, notebook usage, and processing workflow reports. These logs can be configured to be delivered to your cloud storage for further analysis and reporting. Together, the logs allow for security and access auditing, cost optimisations, and identifying high cost users and workloads for further investigation.
Reporting types: API access

Real-time dashboards

Regular reports

Resellers

Supplier type: Reseller providing extra support
Organisation whose services are being resold: Databricks

Staff security

Staff security clearance: Other security clearance
Government security clearance: None

Asset protection

Knowledge of data storage and processing locations: Yes
Data storage and processing locations: United Kingdom

European Economic Area (EEA)

Other locations
User control over data storage and processing locations: Yes
Datacentre security standards: Managed by a third party
Penetration testing frequency: At least every 6 months
Penetration testing approach: Another external penetration testing organisation
Protecting data at rest: Encryption of all physical media
Data sanitisation process: No
Equipment disposal approach: A third-party destruction service

Data importing and exporting

Data export approach: Data resides in the customer's cloud storage account. If any data is saved within Databricks, this can be exported by writing the required code.
Data export formats: CSV

Other
Other data export formats: "Any format supported by Spark "

Parquet

Delta Lake

Avro

JSON

Database tables via ODBC / JDBC connectors

ORC

Any format supported by Pyhton

Any format supported by R
Data import formats: CSV

Other
Other data import formats: DOC

XLS

JSON

PARQUET

ORC

AVRO

Binary

Text

LibSVM

XML

Data-in-transit protection

Data protection between buyer and supplier networks: Private network or public sector network

TLS (version 1.2 or above)
Data protection within supplier network: TLS (version 1.2 or above)

Availability and resilience

Guaranteed availability: Databricks does not publish SLA metrics, but availability notification and detail is available via https://status.databricks.com. Databricks cannot provide SLA details in a questionnaire as they are contractual documents managed through the legal process.
Approach to resilience: "Databricks ensures resilience and availability of its service through a robust architectural framework and strategic datacenter setups. The service operates on a dual-plane system comprising a control plane and compute plane. The control plane manages backend services within the Databricks account, while the compute plane, where data is processed, can be either serverless within the Databricks environment or classic, using resources in the customer's Azure subscription. This structure allows for natural isolation and enhanced security, as the compute resources in the classic plane are within the customer's controlled environment.
For disaster recovery, Databricks employs a comprehensive strategy that includes the use of multiple workspaces and the replication of data across different control planes in different regions. This setup ensures minimal disruption during a disaster, with options for both active-passive and active-active recovery strategies. The active-passive setup involves syncing data from active deployment to a passive one, which can be activated in case of primary region failure. In contrast, an active-active strategy runs processes in both regions simultaneously, offering higher availability but at a higher cost.
This resilient framework ensures that Databricks meets critical business needs but also aligns with best practices for high availability and disaster recovery in cloud services."
Outage reporting: Email alerts and public dashboards. https://status.databricks.com

Identity and authentication

User authentication needed: Yes
User authentication: 2-factor authentication

Identity federation with existing provider (for example Google Apps)

Username or password

Other
Other user authentication: Databricks customers will generally authenticate with their SAML-compliant identity provider, and Databricks supports checks for IP Access Lists or that connectivity occur over private link. Databricks staff must authenticate, MFA, and connect to our VPN to access the production platform.
Access restrictions in management interfaces and support channels: Customers manage the permissions for their own users. For Databricks staff access: Databricks personnel cannot access customer workspaces or production systems except under very specific circumstances. Any access requires authentication via a Databricks-built system that validates access and performs policy checks. Access requires that employees be on our VPN, and our single sign-on solution requires multifactor authentication.
Access restriction testing frequency: At least every 6 months
Management access authentication: 2-factor authentication

Identity federation with existing provider (for example Google Apps)

Username or password

Other
Description of management access authentication: Databricks customers will generally authenticate with their SAML-compliant identity provider, and Databricks supports checks for IP Access Lists or that connectivity occur over private link. Databricks staff must authenticate, MFA, and connect to our VPN to access the production platform.

Audit information for users

Access to user activity audit information: Users have access to real-time audit information
How long user audit data is stored for: User-defined
Access to supplier activity audit information: Users have access to real-time audit information
How long supplier audit data is stored for: User-defined
How long system logs are stored for: At least 12 months

Standards and certifications

ISO/IEC 27001 certification: Yes
Who accredited the ISO/IEC 27001: BDO
ISO/IEC 27001 accreditation date: 18/12/2021
What the ISO/IEC 27001 doesn’t cover: "Whole organisation covered
"
ISO 28000:2007 certification: No
CSA STAR certification: No
PCI certification: Yes
Who accredited the PCI DSS certification: BDO
PCI DSS accreditation date: 25/10/2021
What the PCI DSS doesn’t cover: "Only Available in Azure and AWS.
Available in all regions except Azure Switzerland West.
Serverless workloads are not currently covered for PCI-DSS in Azure.
Most serverless workloads are not currently covered for PCI-DSS. AWS Databricks SQL Serverless workloads are covered under PCI-DSS for AWS us-east-1 and ap-southeast-2 regions only."
Cyber essentials: Yes
Cyber essentials plus: Yes
Other security certifications: No

Security governance

Named board-level person responsible for service security: Yes
Security governance certified: Yes
Security governance standards: ISO/IEC 27001

Other
Other security governance standards: SOC 2 Type II, ISO 27001, ISO 27017, ISO 27018, HIPAA
Information security policies and processes: Databricks has implemented a formal security function with a Chief Security Officer who reports to the CEO and briefs relevant board members. Roles and responsibilities have been defined in Databricks Security Policies. The security policies are based on the ISO 27001 and 27002 standards, and are communicated to relevant parties both annually and upon hire. Databricks Security has defined teams managing compliance, detection and response, security engineering, and offensive security, and works closely with counterparts in legal, vendor management, product security, and engineering. The security program includes responsibility for aligning security to the business and communicating security needs to both the business and Databricks customers.

Operational security

Configuration and change management standard: Supplier-defined controls
Configuration and change management approach: Databricks has a formal, documented change management policy that has been communicated to constituents, approved by management and is reviewed at least once per year. Additionally, within the development process, Databricks requires peer review for every code commit -- commits must be merged by a functional owner for that section of code, and release management is performed by a separate team. Databricks releases require two-person approval, pre- and post-implementation tests, verification for potential operational impacts, rollback plans, documentation, and communications. Depending on the nature of the change, changes may go through a security review and/or approval process.
Vulnerability management type: Supplier-defined controls
Vulnerability management approach: "Databricks has implemented a formal, documented vulnerability management policy and program. The policy is owned by the Databricks CSO, has been approved by management, communicated to constituents, and is reviewed at least annually. Databricks employs an internal team of red hat testers and an external 3rd party to conduct regular penetration testing. We typically perform 8-10 external third-party penetration tests and 15-20 internal penetration tests per year.
Databricks policy requires the remediation or mitigation of critical vulnerabilities within 14 days, high severity vulnerabilities within 30 days, and medium severity vulnerabilities within 60 days."
Protective monitoring type: Supplier-defined controls
Protective monitoring approach: "Databricks employs comprehensive protective monitoring processes. Potential security compromises are detected through advanced analytics and machine learning models, scanning vast amounts of log data to identify anomalies and security threats. Upon detecting potential threats, Databricks' incident response team is engaged immediately to assess and respond. The response is swift, leveraging predefined security protocols and continuous monitoring to ensure that incidents are addressed typically within hours, minimizing potential impact and ensuring rapid resolution. This process underscores Databricks' commitment to operational security and robust protective monitoring.

"
Incident management type: Supplier-defined controls
Incident management approach: Databricks has implemented a formal incident response plan that defines roles, responsibilities, escalation paths, and external communication that is reviewed at least annually. The Databricks Head of Detection and Response is responsible and accountable for managing the incident response program. In addition to the audit logs that are generated from within the Databricks platform (available both to customers and to Databricks Inc for security investigations), Databricks Inc maintains a security incident response team that has access to an internal instance of Databricks that is fed with a wide variety of log sources important to security response including network information.

Secure development

Approach to secure software development best practice: Independent review of processes (for example CESG CPA Build Standard, ISO/IEC 27034, ISO/IEC 27001 or CSA CCM v3.0)

Public sector networks

Connection to public sector networks: No

Social Value

Fighting climate change

We take our environmental management and the impact we have on the environment very seriously. We have environmental policies in place and hold the ISO14001 accreditation. Our environmental assessments are conducted annually by an external Lead ESOS Assessor; they are signed-off by the board and compliance reported to the regulator (the Environment Agency). Our environmental policy is published on our website at https://www.bytes.co.uk/company/sustainability/environmental.
Bytes achieved carbon net zero in March 2022 through approved carbon offsetting schemes. We are always seeking to reduce our impact on the environment. We aim to minimise waste, reduce pollutants and use renewable materials. Our offices have recycling facilities for cans, plastic and paper. We aim to reduce our office printing to zero within the next few years.
An Environmental Steering Committee has been established to coordinate environmental activities and drive change.
To drastically reduce our emissions, we have switched to renewable energy. Our Head Office has reached our first milestone of using a specialist 100% renewable electricity provider. We are also exploring options to install solar panels on our Headquarters building.
Other environmental initiatives include installing electric vehicle charging points and encouraging staff to commute to work without the car (setting up a car share network and installing secure cycle parking).
We produce a SECR (Streamlined Energy and Carbon Reporting) report that details the companies energy consumption and carbon emissions. This report is produced annually by an independent assessor.
This report provides details of our emissions in Scope 1, 2 and 3 categories. It details the activities previously taken to reduce emissions and also recommendations for further improvements.
For scope 1,2 and 3 emissions we aim to reduce these by 50% by 2025-2026 from our 2021 baseline.
We aim to be Net Zero by 2040, covering our own operational emissions.

Pricing

Price: £674 an instance a year
Discount for educational organisations: Yes
Free trial available: Yes
Description of free trial: Databricks Community Edition provides unlimited free access to clusters, a cluster manager ,the notebook environment and integrations for BI analysis. The full Databricks platform offers production-grade functionality, such as an unlimited number of clusters that easily scale up or down, a job launcher, collaboration, advanced security controls, and expert support.
Link to free trial: https://community.cloud.databricks.com/login.html

Databricks Data Intelligence Platform

Features

Benefits

Service documents

Framework

Service ID

Contact

Service scope

User support

Onboarding and offboarding

Using the service

Scaling

Analytics

Resellers

Staff security

Asset protection

Data importing and exporting

Data-in-transit protection

Availability and resilience

Identity and authentication

Audit information for users

Standards and certifications

Security governance

Operational security

Secure development

Public sector networks

Pricing

Service documents

Cookies on Digital Marketplace

Databricks Data Intelligence Platform

Features

Benefits

Pricing

Service documents

Framework

Service ID

Contact

Service scope

User support

Onboarding and offboarding

Using the service

Scaling

Analytics

Resellers

Staff security

Asset protection

Data importing and exporting

Data-in-transit protection

Availability and resilience

Identity and authentication

Audit information for users

Standards and certifications

Security governance

Operational security

Secure development

Public sector networks

Social Value

Pricing

Service documents