Millersoft Ltd

Apache Kafka

Apache Kafka is a distributed event streaming platform that is capable of ingesting and processing trillions of rows per day. It allows a client to handle many different sources and create event queues for a variety of topics whilst scaling to process any amount of data.

Features

  • Scalability - Easily scales vertically and horizontally with zero downtime
  • High-Volume - Handles massive volumes of data easily
  • Data Transformations - Offers provisioning of new data streams
  • Fault Tolerance - Handle failures with zero downtime
  • Reliability - Distributed, partitioned, replicated and fault tolerant
  • Durability - Messages persist on disk super fast
  • Performance - Stable performance at massive throughput and volume
  • Zero Downtime - Guarantees zero downtime and zero data loss
  • Real Time - Kafka handles real time data processing
  • Replication - Through ingest pipelines, it can replicate events

Benefits

  • Deploy in AWS/GCP/Azure, hybrid clouds, Kubernetes, and bare metal
  • Real Time data and results for the user
  • Easily integrates with a variety of tools and software
  • No downtime or data loss so 100% efficiency
  • Can be scaled to meet a clients needs and budget

Pricing

£800 a unit a day

Service documents

Request an accessible format
If you use assistive technology (such as a screen reader) and need versions of these documents in a more accessible format, email the supplier at gerry@millersoftltd.com. Tell them what format you need. It will help if you say what assistive technology you use.

Framework

G-Cloud 13

Service ID

9 6 3 3 2 9 7 1 7 2 5 6 1 6 1

Contact

Millersoft Ltd Gerry Conaghan
Telephone: 0131 376 7114
Email: gerry@millersoftltd.com

Service scope

Software add-on or extension
No
Cloud deployment model
Public cloud
Service constraints
XXX Druid is a write only database for key facts but dimension hierarchies can be configured dynamically out with the Druid application and brought in for analysis via lookups or joins.
System requirements
Linux only

User support

Email or online ticketing support
Email or online ticketing
Support response times
Depends on SLA, normally within 4 hours
User can manage status and priority of support tickets
Yes
Online ticketing support accessibility
None or don’t know
Phone support
Yes
Phone support availability
9 to 5 (UK time), Monday to Friday
Web chat support
No
Onsite support
Yes, at extra cost
Support levels
L1: Tier/Level 1(T1/L1)
Initial support level responsible for basic customer issues. Gathering formation to
determine the issue by analysing the symptoms and figuring out the underlying problem.
L2: Tier/Level 2(T2/L2)
This is a more in-depth technical support level than Tier I containing experienced and more
knowledgeable personnel on a particular product or service.
L3 Tier/Level 3(T3/L3)
Individuals are experts in their fields and are responsible for not only assisting both Tier I and
Tier II personnel, but with the research and development of solutions to new or unknown
issues.
Severity Definitions
1- Critical: Proven Error of the Product in a production environment. The Product Software
is unusable, resulting in a critical impact on the operation. No workaround is available.
2- Serious: The Product will operate but due to an Error, its operation is severely restricted.
No workaround is available.
3- Moderate: The Product will operate with limitations due to an Error that is not critical to
the overall operation. For example, a workaround forces a user and/or a systems
operator to use a time consuming procedure to operate the system; or removes a nonessential
feature.
4- Due to an Error, the Product can be used with only slight inconvenience.
Support available to third parties
Yes

Onboarding and offboarding

Getting started
Documentation and Training
https://kafka.apache.org/quickstart
Service documentation
Yes
Documentation formats
  • HTML
  • PDF
End-of-contract data extraction
All data resides inside the customers cloud/onprem account.
End-of-contract process
Included;
Core Kafka consultancy
Kafka installation and configuration
Kafka tuning
Kafka testing
Security installation and configuration

Excluded;
Kafka support
Kafka upgrades

Using the service

Web browser interface
No
Application to install
No
Designed for use on mobile devices
No
Service interface
No
User support accessibility
None or don’t know
API
No
Customisation available
Yes
Description of customisation
We can accommodate and support custom configuration requests.
User access, roles and authentication are fully configurable.
Cluster size and scale is fully customisable for a clients needs and budget

Scaling

Independence of resources
Kafka can scale horizontally and vertically whilst providing zero downtime and zero data loss.

Analytics

Service usage metrics
No

Resellers

Supplier type
Not a reseller

Staff security

Staff security clearance
Other security clearance
Government security clearance
Up to Developed Vetting (DV)

Asset protection

Knowledge of data storage and processing locations
Yes
Data storage and processing locations
  • United Kingdom
  • European Economic Area (EEA)
User control over data storage and processing locations
Yes
Datacentre security standards
Complies with a recognised standard (for example CSA CCM version 3.0)
Penetration testing frequency
Never
Protecting data at rest
  • Physical access control, complying with CSA CCM v3.0
  • Physical access control, complying with SSAE-16 / ISAE 3402
  • Encryption of all physical media
  • Scale, obfuscating techniques, or data storage sharding
Data sanitisation process
Yes
Data sanitisation type
Deleted data can’t be directly accessed
Equipment disposal approach
Complying with a recognised standard, for example CSA CCM v.30, CAS (Sanitisation) or ISO/IEC 27001

Data importing and exporting

Data export approach
Various export utils via command line
Data export formats
Other
Other data export formats
JSON
Data import formats
Other
Other data import formats
  • Text
  • JSON
  • YAML

Data-in-transit protection

Data protection between buyer and supplier networks
  • TLS (version 1.2 or above)
  • Legacy SSL and TLS (under version 1.2)
  • Other
Other protection between networks
Can also encrypt prior to transfer
Data protection within supplier network
TLS (version 1.2 or above)

Availability and resilience

Guaranteed availability
Customer dependent.
Approach to resilience
AWS services are delivered from multiple datacentres worldwide. When deploying customer services to AWS, Kafka can be configured such that services span multiple availability zones (data centres) to ensure service resilience. Alternatively, our Disaster Recovery as a Service offer can be used to provide DR.
Outage reporting
AWS Cloudwatch alerts can be created

Identity and authentication

User authentication needed
Yes
User authentication
Username or password
Access restrictions in management interfaces and support channels
Access to management interfaces and support channels is restricted through a combination of username and passwords, multifactor authentication, firewalling, IP restrictions, the use of bastion hosts as appropriate.
Access restriction testing frequency
At least once a year
Management access authentication
  • 2-factor authentication
  • Public key authentication (including by TLS client certificate)
  • Username or password

Audit information for users

Access to user activity audit information
Users have access to real-time audit information
How long user audit data is stored for
User-defined
Access to supplier activity audit information
Users have access to real-time audit information
How long supplier audit data is stored for
User-defined
How long system logs are stored for
User-defined

Standards and certifications

ISO/IEC 27001 certification
No
ISO 28000:2007 certification
No
CSA STAR certification
No
PCI certification
No
Cyber essentials
Yes
Cyber essentials plus
No
Other security certifications
No

Security governance

Named board-level person responsible for service security
Yes
Security governance certified
Yes
Security governance standards
Other
Other security governance standards
Cyber Essentials
Information security policies and processes
Millersoft Kafka service follows AWS best practice on security https://aws.amazon.com/security/. We have a range of technical and organisational measures to ensure data security and protection. These cover Access, Roles and Responsibilities, Resource/asset management, Access Control & Authentication, Workstation & Device Security, Network/Communications Security, Back-up, mobile/portable device security, and physical security of our premises. Staff training and awareness is ongoing, staff / contractors must sign confidentiality and privacy statements and read and sign company security policy. Sanctions are applicable for non-compliance. Our reporting structure if a security breach happens or is suspected: staff are trained to and required to immediately flag to DPO and CEO and lock down or isolate the breach where feasible; DPO/CEO will take immediate action including isolation or lock down of affected systems, notification to affected parties, implementation of business continuity and disaster recovery. Risk impact reviews are conducted when a new data category is processed, or system implemented, and security measures adapted as necessary. Category logs, training logs, access logs, and breach logs are maintained, reviewed and signed off periodically by the assigned DPO and CEO.

Operational security

Configuration and change management standard
Supplier-defined controls
Configuration and change management approach
All code is under version control using git
Jenkins is used to build releases
An automated test framework is used for integration testing
Changes are tracked via jira
Cloudformation is used to deploy via AWS Marketplace
Vulnerability management type
Undisclosed
Vulnerability management approach
Solution is deployed into customer's AWS VPC via AWS Cloudformation
External access is configured via customer and GUI is locked down via AWS security groups
SSH access is also locked down via security group and PEM file.
The access is as secure as the customers network.
Patches are in the form of new AWS AMIs
Protective monitoring type
Supplier-defined controls
Protective monitoring approach
All logs go to AWS Cloudwatch for auditing, monitoring and alerting
Incident management type
Supplier-defined controls
Incident management approach
Each instance runs within a VPC within the customers AWS Account. There is no external access or monitoring. Issues need to be reported to the supplier and logs supplied for external analysis.

Secure development

Approach to secure software development best practice
Supplier-defined process

Public sector networks

Connection to public sector networks
No

Social Value

Tackling economic inequality

Tackling economic inequality

In respect of MAC2 (2.2 & 2.3), Millersoft will commit and can demonstrate existing comittment to providing employment and training opportunities to young people, sourced, whenever possible, from the deprived local area. Our recruitment efforts focus on providing local technology students studying in the nearby universities of Edinburgh, Heriot Watt, and Napier with internships that provide work experience, mentoring and training in bleeding edge IT. The ultimate goal of this effort is to provide the internee with full-time employment on completion of their studies. We have contacts in the local universities and liaise over upcoming opportunities. We match these against student skills and goals to identify suitable candidates whom we shortlist for interview. Successful applicants are onboarded, allocated to client projects, given objectives, and a training & development plan that includes skills acquisition related to the project(s) to be worked on, and timescales for the delivery of training and goal attainment. The plan is regularly reviewed and updated by the director or the mentor assigned to the internee. Examples include AWS Practitioner & Technician Certifications, Data Vault Engineer, Apache Druid. Iternees have worked a range of projects including, recently, build of a sales/ ticketing system for a lottery operator in South America, a solution to automate the build of data lakes on AWS, streaming services using Apache Spark and Druid at a card payments processor to process fraud data. As part of any contract tender we will include these initiatives and the methods and processes described to attract, train and retain young employees from our local area. As part skills assessment and subject to specific project needs we can assign internees directly onto projects where they will be mentored by experienced professionals, or if the project requires more experienced staff, bring in internees to provide supervised cover on existing projects.

Pricing

Price
£800 a unit a day
Discount for educational organisations
No
Free trial available
No

Service documents

Request an accessible format
If you use assistive technology (such as a screen reader) and need versions of these documents in a more accessible format, email the supplier at gerry@millersoftltd.com. Tell them what format you need. It will help if you say what assistive technology you use.