Amazon Data Firehose
Amazon Data Firehose provides the easiest way to acquire, transform, and deliver data streams within seconds to data lakes, data warehouses, and analytics services. To use Amazon Data Firehose, you set up a stream with a source, destination, and required transformations.
Features
- Easy launch and configuration.
- Load new data in near real time.
- Elastic scaling to handle varying data throughput.
- Apache Parquet or ORC format conversion.
- Deliver partitioned data to S3.
- Integrated data transformations.
- Support for multiple data destinations.
- Optional automatic encryption.
- Metrics for monitoring performance.
- Pay-as-you-go pricing.
Benefits
- Easily capture, transform, and load streaming data.
- Automatically provision and scale compute, memory, and network resources
- Dynamically partition streaming data without building your own processing pipelines
- Integrated with other AWS services
- Stream data into data lakes and warehouses for analysis
- Enrich data streams with machine learning to analyze data
Pricing
£0 a unit
- Free trial available
Service documents
Request an accessible format
Framework
G-Cloud 14
Service ID
2 2 6 7 7 0 8 7 6 6 3 2 9 2 4
Contact
Softcat Limited
Charles Harrison
Telephone: 01628 403403
Email: psitq@softcat.com
Service scope
- Service constraints
- Please see https://docs.aws.amazon.com/firehose/latest/dev/limits.html
- System requirements
- N/A
User support
- Email or online ticketing support
- Email or online ticketing
- Support response times
-
"Developer Support: General guidance < 24 hours; system impaired < 12 hours.
Business Support: General guidance < 24 hours; system impaired < 12 hours; production system impaired < 4 hours; production system down < 1 hour.
Enterprise Support: General guidance < 24 hours; system impaired < 12 hours; production system impaired < 4 hours; production system down < 1 hour; business-critical system down < 15 minutes
Enterprise On-Ramp Support: Same as Enterprise support, except the response time when business-critical system goes down is < 30 minutes " - User can manage status and priority of support tickets
- Yes
- Online ticketing support accessibility
- None or don’t know
- Phone support
- Yes
- Phone support availability
- 24 hours, 7 days a week
- Web chat support
- Yes, at an extra cost
- Web chat support availability
- 24 hours, 7 days a week
- Web chat support accessibility standard
- None or don’t know
- How the web chat support is accessible
- You can sign in to the AWS Support Center Console at https://console.aws.amazon.com/support/home#/ by using the email address and password associated with your AWS account. To log in with other credentials, see Manage access to AWS Support Center at https://docs.aws.amazon.com/awssupport/latest/user/accessing-support.html.
- Web chat accessibility testing
- None.
- Onsite support
- Yes, at extra cost
- Support levels
-
"Basic Support:
Included for all AWS customers, 24/7 access to customer service, documentation, whitepapers, and AWS re:Post, core “Trusted Advisor” checks, AWS Health dashboard. No designated Technical Account Manager (TAM).
Developer Support:
As above, plus business hours access to Cloud Support Associates, via email (one primary contact). No TAM. Minimum spend of $29 or 3% of monthly AWS usage charges.
Business Support:
As above, plus 24/7 access to Cloud Support Engineers via web, chat and phone (unlimited contacts); Use-case guidance, access to Trusted Advisor checks, AWS Support API; Third-party software support. Access to AWS Countdown Premium for an additional fee. Greater of $100 or spend-dependent percentage (pm).
Enterprise Support:
As above, plus proactive architectural reviews, cost analysis, consultative guidance. Access to TAM, training, AWS Countdown playbooks by AWS experts. For an additional fee, access to AWS Managed Services (AMS), AWS re:Post Private, and AWS Incident Detection and Response. Greater of $15,000 or spend-dependent percentage (pm).
Enterprise On-Ramp Support:
As above, plus concierge access to billing , pool of TAMs, security review, one AWS Countdown each year. AWS Countdown Premium available for an additional fee. Greater of $5,500 or spend-dependent percentage (pm).
" - Support available to third parties
- Yes
Onboarding and offboarding
- Getting started
- AWS provides a range of resources to help customers get started on our services. These include comprehensive documentation (in multiple formats), introductory videos, hands-on labs, online and in-person training, access to a large network of partners and support from the public sector account team.
- Service documentation
- Yes
- Documentation formats
-
- HTML
- Other
- Other documentation formats
- Kindle
- End-of-contract data extraction
- Data may be copied out using AWS API tools to download data.
- End-of-contract process
-
"Buyer may terminate the relationship with Supplier for any reason by (i) providing Supplier with notice and (ii) closing Buyer's account for all services for which Supplier provides an account closing mechanism.
Buyers pay for the services they use to the point of account termination. Please see the AWS UK G-Cloud 14 Pricing Document affiliated with this service in the Digital Marketplace.
Customers retain control and ownership of their data. Supplier will not erase customer data for 30 days following an account termination. This allows customers to retrieve content from Supplier services so long as the customer has paid any charges for any post-termination use of the service offerings and all other amounts due."
Using the service
- Web browser interface
- Yes
- Using the web interface
-
"The AWS Management Console provides a simple web interface for Amazon Web Services. Users can log in with your AWS account name and password. If you’ve enabled AWS Multi-Factor Authentication, you will be prompted for your device’s authentication code. Almost all functionality for each of our services is exposed through the AWS Management Console. It facilitates management for all aspects of the AWS account in a consolidated view whilst providing access to all services and their respective functionalities.
In some cases, specific configuration parameters of a service are dedicated to, and only available from, the AWS Command Line Interface, AWS SDK, or the API." - Web interface accessibility standard
- None or don’t know
- How the web interface is accessible
- You can sign in to the Support Center at https://console.aws.amazon.com/support/home#/ using your email address and password associated with your AWS account. To log in with other credentials, see Manage access to AWS Support Center at https://docs.aws.amazon.com/awssupport/latest/user/accessing-support.html.
- Web interface accessibility testing
- None.
- API
- Yes
- What users can and can't do using the API
- All functionality is exposed via an API.
- API automation tools
-
- Ansible
- Chef
- SaltStack
- Terraform
- Puppet
- Other
- Other API automation tools
-
- Wide range of third party services work with AWS APIs.
- SDKs for Python, Ruby, PHP, JavaScript, Java, .NET, Node.js
- API documentation
- Yes
- API documentation formats
-
- HTML
- Other
- Command line interface
- Yes
- Command line interface compatibility
-
- Linux or Unix
- Windows
- MacOS
- Using the command line interface
- All functionality is available via the AWS CLI.
Scaling
- Scaling available
- Yes
- Scaling type
- Automatic
- Independence of resources
-
"Customer environments are logically segregated to prevent users and customers from accessing resources not assigned to them.
Services which provide virtualized operational environments to customers (i.e. EC2) ensure that customers are segregated via security management processes/controls at the network and hypervisor level.
AWS continuously monitors service usage to protect infrastructure and support availability commitments/requirements. AWS maintains a capacity planning model to assess infrastructure usage and demands at least monthly, and usually more frequently. The AWS capacity planning model supports the planning of future demands to acquire and implement additional resources based on current resources and forecasted requirements." - Usage notifications
- Yes
- Usage reporting
-
- API
- SMS
- Other
- Other usage reporting
- As per service description.
Analytics
- Infrastructure or application metrics
- Yes
- Metrics types
- Other
- Other metrics
-
- Data Partition metrics
- Data Delivery Metrics
- Data Ingestion Metrics
- Data Transformation Metrics
- Format Conversion Metrics
- Encryption Metrics
- Reporting types
-
- API access
- Real-time dashboards
Resellers
- Supplier type
- Reseller (no extras)
- Organisation whose services are being resold
- AWS
Staff security
- Staff security clearance
- Conforms to BS7858:2019
- Government security clearance
- Up to Developed Vetting (DV)
Asset protection
- Knowledge of data storage and processing locations
- Yes
- Data storage and processing locations
-
- United Kingdom
- European Economic Area (EEA)
- Other locations
- User control over data storage and processing locations
- Yes
- Datacentre security standards
- Supplier-defined controls
- Penetration testing frequency
- At least every 6 months
- Penetration testing approach
- Another external penetration testing organisation
- Protecting data at rest
- Other
- Other data at rest protection approach
-
"AWS adheres to independently validated privacy, data protection, security protections and control processes. These are listed under “standards and certifications”.
AWS is responsible for the security of the cloud; customers are responsible for security in the cloud. AWS enables customers to control their content (where it will be stored, how it will be secured in transit or at rest, how access to their AWS environment will be managed).
Wherever appropriate, AWS offers options to add additional security layers to data at rest, via scalable and efficient encryption features. AWS offers flexible key management options and dedicated hardware-based cryptographic key storage" - Data sanitisation process
- Yes
- Data sanitisation type
-
- Explicit overwriting of storage before reallocation
- Deleted data can’t be directly accessed
- Hardware containing data is completely destroyed
- Equipment disposal approach
- In-house destruction process
Backup and recovery
- Backup and recovery
- Yes
- What’s backed up
- Use Amazon S3 to backup all or failed only data
- Backup controls
- You can specify the S3 backup settings for your Kinesis Data Firehose delivery stream depending on the configuration choices made.
- Datacentre setup
- Multiple datacentres with disaster recovery
- Scheduling backups
- Supplier controls the whole backup schedule
- Backup recovery
- Users contact the support team
Data-in-transit protection
- Data protection between buyer and supplier networks
-
- Private network or public sector network
- TLS (version 1.2 or above)
- IPsec or TLS VPN gateway
- Bonded fibre optic connections
- Legacy SSL and TLS (under version 1.2)
- Data protection within supplier network
- Other
- Other protection within supplier network
-
"Customer environments are logically segregated to prevent users and customers from accessing resources not assigned to them. AWS gives customers ownership and control over their content by design through simple, but powerful tools that allow customers to determine how their content will be secured in transit.
AWS enables customers to open a secure, encrypted channel to AWS services using TLS/SSL, and/or IPsec or TLS VPN (if applicable), or other means of protection the customers wish to use.
API calls can be encrypted with TLS/SSL to maintain confidentiality; the AWS Console connection is encrypted with TLS."
Availability and resilience
- Guaranteed availability
-
"AWS currently provides SLAs for several services. Due to the rapidly evolving nature of AWS’s product offerings, SLAs are best reviewed directly on our website via the link below: https://aws.amazon.com/legal/service-level-agreements/
Well-architected solutions on AWS that leverage AWS service SLAs and unique AWS capabilities such as multiple Availability Zones, can ease the burden of achieving specific SLA requirements. We encourage customers to visit our well architected documentation on our website, at https://docs.aws.amazon.com/wellarchitected/latest/reliability-pillar/appendix-a-designed-for-availability-for-select-aws-services.html" - Approach to resilience
-
"The AWS Business Continuity plan details the process that AWS follows in the case of an outage, from detection to deactivation. AWS has developed a three-phased approach: Activation and Notification Phase, Recovery Phase, and Reconstitution Phase. This approach ensures that AWS performs system recovery and reconstitution efforts in a methodical sequence, maximizing the effectiveness of the recovery and reconstitution efforts and minimizing system outage time due to errors and omissions.
AWS maintains a ubiquitous security control environment across all regions. Each data centre is built to physical, environmental, and security standards in an active-active configuration, employing an n+1 redundancy model, ensuring system availability in the event of component failure. Components (N) have at least one independent backup component. All data centres are online and serving traffic. In case of failure, there is sufficient capacity to enable traffic to be load-balanced to the remaining sites.
Customers are responsible for implementing contingency planning, training and testing for their systems hosted on AWS. AWS provides customers with the capability to implement a robust continuity plan, including the frequent server instance back-ups, data redundancy replication, and the flexibility to place instances and store data within multiple geographic regions across multiple Availability Zones." - Outage reporting
- Public dashboard; personalised dashboard with API and events; configurable alerting (email/SMS/messaging)
Identity and authentication
- User authentication
-
- 2-factor authentication
- Identity federation with existing provider (for example Google apps)
- Limited access network (for example PSN)
- Dedicated link (for example VPN)
- Username or password
- Other
- Other user authentication
- AWS’s Identity and Access Management (IAM) system allows you to control access to AWS services/resources. No actions are permissible without authentication. IAM facilitates the issuance of access permissions per user/group. MFA is available at no extra cost. AWS CloudTrail allows you to log, continuously monitor and retain events related to API calls across your AWS infrastructure.
- Access restrictions in management interfaces and support channels
- AWS IAM provides user access control to AWS services, APIs and specific resources. Other controls include time, originating IP address, SSL use, and whether users authenticated via Multi Factor Authentication devices. API calls to launch/terminate instances, change firewalls, and perform other functions are signed by customers’ Amazon Secret Access Key (either the root AWS Account’s Secret Access Key or the Secret Access key of a user created with IAM). API calls can be encrypted with TLS/SSL for confidentiality and customers can use TLS/SSL-protected API endpoints.
- Access restriction testing frequency
- At least every 6 months
- Management access authentication
-
- 2-factor authentication
- Other
- Description of management access authentication
- AWS controls access to systems through authentication that requires a unique user ID and password. AWS systems don't allow actions to be performed without identification or authentication. Remote access requires multi-factor authentication. All remote administrative access attempts are logged and reviewed by the Security team for unauthorized attempts or suspicious activity. Incident response procedure is triggered in case of suspicious activity. AWS employs the concept of least privilege, allowing only the necessary access for users to accomplish their job. User access to AWS systems requires documented approval from authorized personnel.
- Devices users manage the service through
- Dedicated device on a segregated network (providers own provision)
Audit information for users
- Access to user activity audit information
- Users have access to real-time audit information
- How long user audit data is stored for
- User-defined
- Access to supplier activity audit information
- You control when users can access audit information
- How long supplier audit data is stored for
- At least 12 months
- How long system logs are stored for
- At least 12 months
Standards and certifications
- ISO/IEC 27001 certification
- Yes
- Who accredited the ISO/IEC 27001
- EY CertifyPoint
- ISO/IEC 27001 accreditation date
- 22/11/2023
- What the ISO/IEC 27001 doesn’t cover
- N/A
- ISO 28000:2007 certification
- No
- CSA STAR certification
- Yes
- CSA STAR accreditation date
- 22/11/2023
- CSA STAR certification level
- Level 2: CSA STAR Attestation
- What the CSA STAR doesn’t cover
- N/A
- PCI certification
- Yes
- Who accredited the PCI DSS certification
- Coalfire Systems Inc.
- PCI DSS accreditation date
- 13/12/2023
- What the PCI DSS doesn’t cover
- N/A
- Cyber essentials
- Yes
- Cyber essentials plus
- Yes
- Other security certifications
- Yes
- Any other security certifications
-
- ISO 27017
- ISO 27018
- SOC 1/2/3
Security governance
- Named board-level person responsible for service security
- Yes
- Security governance certified
- Yes
- Security governance standards
-
- ISO/IEC 27001
- Other
- Other security governance standards
- ISO 27017, ISO27018, Cyber Essentials Plus, SOC1/2/3
- Information security policies and processes
-
"AWS implements formal, documented policies and procedures that provide guidance for operations and information security within the organisation. Policies address purpose, scope, roles, responsibilities and management commitment.
Employees maintain policies in a centralised and accessible location. AWS Security Assurance is responsible for familiarizing employees with the AWS security policies.
AWS has established information security functions that are aligned with defined structure, reporting lines, and responsibilities. Leadership involvement provides clear direction and visible support for security initiatives.
Policies are approved by AWS leadership at least annually or following a significant change to the AWS environment."
Operational security
- Configuration and change management standard
- Supplier-defined controls
- Configuration and change management approach
-
"Changes to AWS services and features follow secure software development practices, including security risk reviews prior to launch. Developer access to production environments is via explicit access system requests, subject to owner review and authorisation.
Teams set bespoke change management standards per service, underpinned by standard AWS guidelines.
All production environment changes are reviewed, tested and approved. Stages include design, documentation, implementation (including rollback procedures), testing (non-production environment), peer to peer review (business impact/technical rigour/code), final approval by authorised party.
Emergency changes follow AWS incident response procedures. Exceptions to change management processes are documented and escalated to AWS management. " - Vulnerability management type
- Supplier-defined controls
- Vulnerability management approach
-
"AWS Security performs vulnerability scans on the host operating system, web applications, and databases in the AWS environment. Approved third party vendors conduct external assessments (minimum frequency: quarterly). Identified vulnerabilities are monitored and evaluated. Countermeasures are designed and implemented to neutralise known/newly identified vulnerabilities.
AWS Security monitors newsfeeds/vendor sites for patches and receives customer intelligence via http://aws.amazon.com/security/vulnerability-reporting/.
Security and Compliance is a shared responsibility between AWS and the customer, and the customer must perform some of the vulnerability management process. See https://aws.amazon.com/compliance/shared-responsibility-model/ for more details." - Protective monitoring type
- Supplier-defined controls
- Protective monitoring approach
-
"AWS deploys (pan-environmental) monitoring devices to collect information on unauthorized intrusion attempts, usage abuse, and network/application bandwidth-usage. Devices monitor:
• Port scanning attacks
• Usage (CPU, processes, disk utilization, swap rates, software-error generated losses)
• Application metrics
• Unauthorized connection attempts
AWS security controls are reviewed by independent external auditors during audits for our SOC, PCI DSS and ISO 27001 compliance.
Security and Compliance is a shared responsibility between AWS and the customer. See https://aws.amazon.com/compliance/shared-responsibility-model/ for more details." - Incident management type
- Supplier-defined controls
- Incident management approach
-
"AWS adopts a three-phased approach to manage incidents:
1. Activation and Notification Phase
2. Recovery Phase
3. Reconstitution Phase
To ensure the effectiveness of the AWS Incident Management plan, AWS conducts incident response testing, providing excellent coverage for the discovery of defects and failure modes as well as testing the systems for potential customer impact.
The Incident Response Test Plan is executed annually, in conjunction with the Incident Response plan. It includes multiple scenarios, potential vectors of attack, the inclusion of the systems integrator in reporting and coordination and varying reporting/detection avenues.
"
Secure development
- Approach to secure software development best practice
- Independent review of processes (for example CESG CPA Build Standard, ISO/IEC 27034, ISO/IEC 27001 or CSA CCM v3.0)
Separation between users
- Virtualisation technology used to keep applications and users sharing the same infrastructure apart
- Yes
- Who implements virtualisation
- Supplier
- Virtualisation technologies used
- Other
- Other virtualisation technology used
- AWS proprietary
- How shared infrastructure is kept separate
- Customer environments are logically segregated, preventing users and customers from accessing unassigned resources. Customers maintain full control over their data access. Services which provide virtualized operational environments to customers, ensure that customers are segregated and prevent cross-tenant privilege escalation and information disclosure via hypervisors and instance isolation.
Energy efficiency
- Energy-efficient datacentres
- Yes
- Description of energy efficient datacentres
- AWS is committed to running our business in the most environmentally friendly way possible. In addition to the environmental benefits inherently associated with running applications in the cloud, AWS is focused on efficiency and continuous innovation across our global infrastructure. We are committed to powering our operations with 100% renewable energy by 2025. New servers purchased to host the cloud hosting for AWS data centers in the European Union (EU) meet the requirements of EU 2019/424 (“Lot 9"). You can read more on our website https://sustainability.aboutamazon.com/environment/the-cloud
Social Value
- Social Value
-
Social Value
- Fighting climate change
- Tackling economic inequality
- Equal opportunity
- Wellbeing
Fighting climate change
Softcat are dedicated to reducing our environmental impact and actively promoting sustainability. Our commitment to sustainability is a core aspect of our business strategy, driving us to innovate and lead in the creation of a more sustainable future for our company and the communities we serve. This commitment is embedded in our policies, operating procedures, and training programs.
We are proud to be the first FTSE 250 company to be awarded 5-star status in relation to the United Nations Sustainable Development Goals.
We aim to achieve a Carbon Net-Zero Value Circle by 2040 by prioritising renewable energy, reducing natural resource use, minimising waste, and safeguarding biodiversity in compliance with environmental legislation.
At Softcat, we have taken significant steps towards securing renewable energy across our organisation, reducing our scope 1 & 2 emissions. We had the target of using 100% Renewable Energy across all our locations by 2024. We successfully delivered against this target ~2 years early.
In May 2023 we took delivery of 15 electric vehicles, replacing all existing fossil-fuelled company cars used by employees for business means. The implementation of the EV pool fleet will see a saving of over 80 tons of CO2e per year. A huge impact on our Net Zero targets.Tackling economic inequality
As a value-add reseller, Softcat outsources the products, services, and solutions through our extensive network of partners, to best suit the needs of our broad client base. We always consider and promote SMEs and local providers where appropriate, particularly for the products and services we offer via the G Cloud framework.
We remain dedicated to improving employability and educational awareness across schools, colleges, and universities to help break down the barriers to joining technology organisations.
We work collaboratively with many schools that are close in proximity to our offices, to ensure we are actively supporting the community as well as schools from lower socio-economic backgrounds.
We visit the schools to talk about the IT sector and the roles in our organisation, as well as promoting work-experience opportunities during the summer. In particular, we actively encourage students from diverse backgrounds to engage in work experience to appreciate the roles available in our sector.
For ambitious school and college leavers, a Softcat Apprenticeship is a great first step into the world of work, with 94% of our apprentices offered a permanent position at Softcat post apprenticeships, which goes to show the amazing opportunity available with us.
We were ranked 1st in IT & Consultancy, and 10th overall in by RateMyApprenticeship.com - Best 100 Apprenticeship Employers 2023-2024 list.
Softcat now also offer 12 month paid internships to University students looking to complete a year in industry as part of their undergraduate studies.Equal opportunity
Our approach to diversity and inclusion is introduced first during our induction training, as part of our Softcat values, outlining responsibility to uphold our principles. This message is reinforced by our process and policies, networks, Allyship Training and Inclusion Awareness campaigns.
Softcat supports diversity and inclusion through various networks including:
- Supporting Women in Business (SWIB)
- The Ethnic and Cultural Network
- The Pride Network
- The Family Network
- The Empowering Disability and Neurodiversity Network (EDN)
- The Faith at Work Network
- Armed Forces & Veterans Network
These networks aim to create a supportive and inclusive work environment for all employees, regardless of gender, ethnicity, sexual orientation, disability, or family commitments.
Our allyship programme, Stronger Together, is a mixture of event and workshop-based training available to all staff. Programme topics include, bias, power, privilege, and being a greater ally.
Inclusion Awareness campaigns include race, disability, sexual orientation, gender, faith, and caring responsibilities. These sessions highlight and celebrate minority groups, through panel sessions, Q&A sessions and training, providing an opportunity to discuss and understand ways to be more inclusive.
Our efforts to improve diversity and inclusion have been incredibly successful. Since 2020, the number of female employees below management level has increased to 35%, and the number of ethnic minority employees rose to 17%.Wellbeing
At Softcat, all employees are provided with access to our multidimensional wellbeing programme which includes flexible work arrangements, free nutritious breakfast, mental health support, employee benefits scheme, health and wellbeing week activities, and online workshops.
Giving back to the community is an innate part of who we are as a company. All Softcat employees are therefore given two volunteer days per year to support a charitable or community cause.
Each of our 10 regional offices also support local charities through fundraising, donations and events. For example, our Manchester office has raised over £30,000 for the WeLoveMCR charity. This funding has supported young, disadvantaged Manchester citizens in gaining qualifications to broaden their work opportunities and supporting local groups in delivering indispensable services that enable community cohesion.
Pricing
- Price
- £0 a unit
- Discount for educational organisations
- No
- Free trial available
- Yes
- Description of free trial
- Please see AWS Free Trail