Azure Cosmos DB approaches data consistency as a spectrum of choices. This approach includes more options than the two extremes of strong and eventual consistency. You can choose from five well-defined levels on the consistency spectrum.
With Cosmos DB any write into any region must be replicated and committed to all configured regions within the account.
Reference:
https://docs.microsoft.com/en-us/azure/cosmos-db/consistency-levels
App1 data is available to Databricks
Azure Data Lake Storage
Azure Data Factory
Automate data movement using Azure Data Factory, then load data into Azure Data Lake Storage, transform and clean it using Azure Databricks, and make it available for analytics using Azure Synapse Analytics. Modernize your data warehouse in the cloud for unmatched levels of.
Note:
Integrate data silos with Azure Data Factory, a service built for all data integrations needs and skills level.
Easily Construct ETL and ELT processes code-free within the intuitive visual environment, or write your own code. Visually integrate data sources using more than 90+ natively built and maintainence-free connectors at no added cost.
Focus on your data-- the serverless integration service does the rest.
Reference:
https://azure.microsoft.com/en-us/services/databricks/#capabilities
https://azure.microsoft.com/en-us/services/data-factory/
From the Azure Active Directory admin center, register app1, from the access control ( IAM ) blade, delegate permissions.
The app must be registered. You can register the application in the Azure Active Directory admin center.
The Azure AD access reviews feature has an API in the Microsoft Graph endpoint.
you can register an Azure AD applications and set it up for permissions to call the access reviews API in graph.
https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app
Azure Storage and Azure SQL Database linked services contain connection strings that Data Factory uses at runtime to connect to your Azure Storage and Azure SQL Database
Dataset:
A dataset is a named view of data that simply points or references the data you want to use in your activities as inputs and outputs.
Datasets identify data within different data stores, such as tables, files, folders and documents.
For example:
An Azure Blob dataset specifies the blob container and folder in Blob storage from which activity should read the data.
https://docs.microsoft.com/en-us/azure/data-factory/concepts-datasets-linked-services
You can copy data from Azure Blob to Azure SQL Database using Azure Data Factory.
Restore point:
In the vault associated with the VM you want to restore, click Backup items > Azure virtual machine.
Click a VM. By default on the VM dashboard, recovery points from the last 30 days are displayed.
You can display recovery points older than 30 days or filter to find recovery points based on dates, time ranges and different types of snapshot consistency.
To restore the VM, click Restore VM.
Reference:
https://docs.microsoft.com/en-us/azure/backup/backup-azure-arm-restore-vms
Azure Automation now supports update management, inventory and change tracking.
Update management delivers visibility of update compliance across Azure, on-premises, and other clouds for both Windows and Linux. Create scheduled deployments to orchestrate the installation of updates within a defined maintenance window. Exclude specific updates and get troubleshooting logs to identify any issues during the deployment.
We can use the Azure Premium Key Vault with Hardware Security Modules ( HSM ) backed keys.
The Key Vault has to be in the same region as the VM that will be encrypted.
Note: If you want to use a key encryption key ( KEK ) for an additional layer of security for encryption keys, add a KEK to your key vault. Use the Add-AzKeyVaultKey cmdlet for create a key encryption key in the key vault. You can also import a KEK from your on-premises key management HSM.
Reference:
https://docs.microsoft.com/en-us/azure/security/fundamentals/azure-disk-encryption-vms-vmss
Automatic tuning : helps to improve the performance of the DB.
Azure SQL Database and Azure SQL Managed Instance automatic tuning provides peak performance and stable workloads through continuous performance tuning based on AI and machine learning.
Automatic tuning is a fully managed intelligent performance service that uses built-in intelligence to continously monitor queries executed on a database and it automatically improves their performance.
Reference:
https://docs.microsoft.com/en-us/azure/azure-sql/database/automatic-tuning-overview
Install a self-hosted integration runtime
The integration runtime is a customer-managed data integration infrastructure used by Azure Data Factory to provide data integration capabilities across different network environments.
Create a pipeline:
With ADF, existing data processing services can be composed into data pipelines that are highly available and managed in the cloud.
These data pipelines can be scheduled to ingest, prepare , transform, analyze and publish data and ADF manages and orchestrates the complex data and processing dependencies.
https://docs.microsoft.com/en-us/azure/machine-learning/team-data-science-process/move-sql-azure-adf
How to create and configure a self-hosted integration runtime ?
Azure Data Factory service: to copy data from the server onto Azure Storage.
why to install the self-hosted integration runtime?
You have to install the self-hosted integration runtime on the server so that it can be accepted as a data source in Azure Data Factory.
create and configure a self-hosted integration runtime
The integration runtime ( IR ) is the compute infrastructure that Azure Data Factory uses to provide data-integration capabilities across different network environments.
A self-hosted integration runtime can run copy activities between a cloud data store and a data store in a private network. It can dispatch transform activities against compute resources in an on-prem network or an Azure virtual network. The installation of a self-hosted integration runtime needs an on-prem machine or a virtual machine inside a private network.
Immutable storage for Azure Blob storage enables users to store business-critical data objects in a WORM ( Write Once, Read Many ) state.
Immutable storage supports:
Time-based retention policy support: Users can set policies to store data for a specified interval.
When a time-based retention policy is set, blobs can be created and read, but not modified or deleted. After the retention period has expired, blobs can be deleted but not overwritten.
References:
https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-immutable-storage
Note:
Immutable storage for Azure Blob storage enables users to store business-critical data objects in a WORM ( write once, read many ) state. This state makes the data non-erasable and non-modifiable for a user-specified interval. For the duration of the retention interval, blobs can be created and read, but cannot be modified or deleted. Immutable storage is available for general-purpose v2 and Blob storage accounts in all azure regions.
https://azure.codefari.com/2018/12/what-are-availability-set-fault-domain.html
Identity Protection uses adaptive machine learning algorithms and heuristics to detect anomalies and risk detections that might indicate that an identity has been compromised. Using this data, Identity protection generates reports and alerts so that you can investigate these risk detections and take appropriate remediation or mitigation action.
Reference:
https://docs.microsoft.com/en-us/azure/security/fundamentals/threat-detection
You can create a managed identity for App Service and Azure Functions applications and how to use it to access other resources.
A managed identity from Azure Active Directory ( AAD ) allows your app to easily access other AAD-protected resources such as Azure Key Vault.
References:
https://docs.microsoft.com/en-us/azure/app-service/overview-managed-identity?tabs=dotnet
others
Use the authentication-managed-identity policy to authenticate with a backend service using the managed identity of the API management service. This policy essentially uses the managed identity to obtain an access token from Azure Active Directory for accessing the specified resource.
After successfully obtaining the token, the policy will set the value of the token in the authorization header using the bearer scheme.
Which reference material should you use when designing the app ?
Microsoft's obligation to accessibility is guided by three main principles: transparency, inclusivity and accountability. In developing our products and services, we take into account leading global accessibility standards, including:
EN 301 549
E.S Section 508
Web Content Accessibility Guidelines ( WCAG )
References:
https://www.microsoft.com/en-us/trust-center/compliance/accessibility
Data Migration Assistant :
Data Migration Assistant is used to migrate SQL databases.
Migrate the virtual machines to an Azure Subscription:
Azure Site Recovery
Site Recovery can replicate on-prem VMWare VMs, Hyper-V VMs, physical servers ( Windows and Linux ), Azure Stack VMs to Azure.
Note:
Site Recovery helps ensure business continuity by keeping business apps and workloads running during outages, Site Recovery replicates workloads running on physical and virtual machines ( VMs ) from a primary site to a secondary location, and access apps from there. After the primary location is running again, you can fail back to it.
https://docs.microsoft.com/en-us/azure/site-recovery/site-recovery-overview
Copy the files:
AzCopy only copy files, not the disks.
Microsoft initiated a powerful solution that helps customers get their data to the Azure Public Cloud in a cost-effective , secure, and efficient manner with powerful Azure and machine learning at play.
The solution is called Data box.
Data Box and is in general availability status. It is a rugged device that allows organizations to have 100 TB of capacity on which to copy their data and then send it to be transferred to Azure.
A company named A, Ltd. has an Azure Active Directory ( Azure AD ) tenant that is integrated with Microsoft Office 365 and an Azure Subscription.
A Ltd has an on-premises identity infrastructure. The infrastructure includes servers that run Active Directory Domain Services ( AD DS ), Active Directory Federation Services ( AD FS ), Azure AD connect and Microsoft Identity Manager ( MIM )
A Ltd has a partnership with a company named B, Inc . B has an Active Directory forest and an Office 365 tenant. B Inc has the same on-prem identity infrastructure as A Ltd
A team of 20 developers from B Inc will work on an Azure solution that will be hosted in the Azure Subscription of A Ltd. The developers must be added to the contributor role for a resource in the A Ltd
we have to ensure that A Ltd can assign the role to the 20 B Inc developers.
The solution must ensure that the B Inc developers use their existing credentials to access resources.
Preferred solution:
Configure a forest trust between the on-premises Active Directory forests of A Ltd and B Inc.
What is meant by trust configuration ?
Trust Configurations: Configure trust from managed forests or domains to the administrative forest A one-way trust is required from production environment to the admin forest.
Selective authentication should be used to restrict accounts in the admin forest to only logging on to the appropriate production hosts.
Reference:
https://docs.microsoft.com/en-us/windows-server/identity/securing-privileged-access/securing-privileged-access-reference-material
Example :
you have 10 Standard_F2S_v2 Azure virtual machines.
Each virtual machine has two network adapters.
Preferred solution:
Enable Accelerated Networking
Accelerated networking enables single root I/O virtualization ( SR-IOV ) to a VM, greatly improving its networking performance. This high-performance path bypasses the host from the datapath, reducing latency, jitter and CPU utilization, for use with the most demanding network workloads on supported VM types.
Reference:
https://docs.microsoft.com/en-us/azure/virtual-network/create-vm-accelerated-networking-cli
preferred service:
Azure file share and Azure file sync
Use Azure file sync to centralize your organisations file share in Azure files, while keeping the flexibility, performance and compatibility of an on-premises file server.
Azure file sync transforms windows server into a quick cache of your Azure File Share.
You need an Azure file share in the same region that you want to deploy Azure file sync.
Reference:
https://docs.microsoft.com/en-us/azure/storage/files/storage-sync-files-deployment-guide?tabs=azure-portal%2Cproactive-portal
Preferred cloud service : Azure Service Bus
Asynchronous messaging can be implemented in a variety of different ways. With queues, topics, and subscriptions, Azure Service Bus supports asynchronism via a store and forward mechanism.
Service Bus is a transactional message broker and ensures transactional integrity for all internal operations against its message stores. All transfers of messages inside of Service Bus, such as moving messages to a dead-letter queue or automatic forwarding of messages between entities, are transactional.
Reference:
https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-async-messaging
https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-transactions
We have a static content files on the Azure Web App.
Configure a CNAME DNS record for the Azure Content Delivery Network ( CDN ) domain.
Place the static content in Azure Blob storage and enable Content Delivery Network ( CDN ) on the account.
Reference:
https://docs.microsoft.com/en-us/azure/cdn/cdn-map-content-to-custom-domain?tabs=azure-dns
https://docs.microsoft.com/en-us/azure/cdn/cdn-add-to-web-app
Ability to encrypt operating system disks and data disks.
For enhanced virtual machines ( VM ) security and compliance, virtual disks in Azure can be encrypted.
Disks are encrypted by using cryptographic keys that are secured in an Azure Key Vault. You control these cryptographic keys and can audit their use.
Reference:
https://docs.microsoft.com/en-us/azure/virtual-machines/windows/disk-encryption-overview
Syslog table for the queries on events from Linux Virtual machines.
Note: Syslog is an event logging protocol that is common to Linux. Applications will send messages that may be stored on the local machine or delivered to a Syslog collector. When the Log Analytics agent for linux is installed, it configures the local Syslog daemon to forward messages to the agent. The agent then sends the message to Azure Monitor where a corresponding record is created.
Reference:
https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-sources-syslog
Need to recommend a solution that meets the following requirements:
1. Minimize the use of the virtual machine processors to transfer data.
2. Minimizes network latency
Virtual machine size : High performance compute Standard_H16r.
Feature : Remote Direct Memory Access ( RDMA ).
https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-hpc#h-series
A Conditional access policy and two named locations.
Conditional Access Policies are at their most basic an if-then statement combining signals, to make decisions and enforce organization policies. One of those signals that can be incorporated into the decision making process is network location.
https://docs.microsoft.com/en-us/azure/active-directory/conditional-access/location-condition#named-locations
To be able to create an alert we send the Azure AD logs to an Azure Log Analytics workspace.
Signal type : Log
( Ensure Resource Type is an analytics source like log analytics or application insights and signal type as log ).
https://docs.microsoft.com/en-us/azure/azure-monitor/platform/alerts-log
Example : Have an Azure Subscription that contains 200 Azure Virtual machines that runs Windows Server 2016
You need to Centrally monitor all warning events in the System logs of the virtual machines?
The Map feature in Azure Monitor for VM gets its data from the Microsoft Dependency agent.
The Dependency agent relies on the Log Analytics agent for its connection to Log Analytics. So your system must have the Log Analytics agent installed and configured with the Dependency agent.
Whether you enable Azure Monitor for VMs for a Single Azure VM on you use the at-scale deployment method, use the Azure VM Dependency agent extension to install the agent as part of the experience.
In a hybrid environment, you can download and install the Dependency agent manually. If your VMs are hosted outside Azure, use an automated deployment method configuration to perform on the virtual machines :
Enable Virtual machines scale set to setup Azure Monitor for VMs:
Enable a single Azure VM or vmss by selecting Insights ( preview ) directly from the VM or virtual machine scale set.
Enable two or more Azure VMs and vmss by using Azure Policy. This method ensures that an existing and new VMs and scalesets, the required dependencies are installed and properly configured.
Noncompliant VMs and scale sets are reported, so you can decide whether to enable them and to remediate them.
Enable two or more azure vms or vmss across a specified subscription or resource group by using powershell.
https://docs.microsoft.com/en-us/azure/azure-monitor/insights/vminsights-enable-overview
Use Azure Batch to run large-scale parallel and high-performance computing ( HPC ) batch jobs efficiently in Azure. Azure Batch creates and manages a pool of compute nodes ( virtual machines ), installs the applications you want to run and schedules jobs to run on the nodes.
There's no cluster or job scheduler software to install, manage or scale. Instead , you use Batch APIs and tools, command-line scripts or the Azure portal to configure , manage and monitor your jobs.
Developers can use Batch as a platform service to build SaaS applications or client apps where large-scale execution is required.
=========================================================================
Azure Batch works well with intrinsically parallel ( also known as " embarrassingly parallel " ) workloads.
Intrinsically parallel workloads are those where the applications can run independently and each instance completes part of the work. When the applications are executing, they might access some common data, but they do not communicate with other instances of the application.
Intrinsically parallel workloads can therefore run at a large scale determined by the amount of compute resources available to run applications simultaneously.
Scenario : The application consumes data from multiple databases.
Application code references database tables using a combination of the server, database and table name.
How to migrate the application instance to azure ?
SQL Server Stretch Database.
Access your SQL Server data seamlessly regardless of whether its on-prem or stretched to the cloud. You set the policy that determines where data is stored and SQL Server handles the data movement in the background. The entire table is always online and queryable. And, Stretch Database does not require any changes to existing queries or applications
The location of the data is completely transparent to the application.
SQL Managed Instance.
The managed instance deployment model is designed for customers looking to migrate a large number of apps from on-prem or Iaas , self-built or ISV provided environment to fully managed PaaS cloud environment, with as low migration effort as possible. Using the fully automated Data Migration Service ( DMS ) in Azure, customers can lift and shift their on-prem SQL server to a managed instance that offers compatibility with SQL server on-prem and complete isolation of customer instances with native VNET support.
https://docs.microsoft.com/en-us/sql/sql-server/stretch-database/stretch-database?view=sql-server-ver15
https://docs.microsoft.com/en-us/azure/azure-sql/managed-instance/sql-managed-instance-paas-overview
The Network Watcher Network Performance monitor is a cloud-based hybrid network monitoring solution that helps you monitor network performance between various points in your network infrastructure. It also helps you monitor network connectivity to service and application endpoints and monitor the performance of Azure Express Route.
Note:
IP flow verify checks if a packet is allowed or denied to or from a virtual machine. The information consists of direction, protocol , local IP , remote IP , local port and remote port. If the packet is denied by a security group, the name of the rule that denied the packet is returned. While any source or destination IP can be chosen, IP flow verify helps administrator quickly diagnose connectivity issues from or to the internet and from or to the on-premises environment.
IP flow verify looks at the rules for all NSG appiled to the network interface, such as a subnet or virtual machine NIC. Traffic flow is then verified based on the configured settings to or from that network interface.
IP flow verify is useful in confirming if a rule in a NSG is blocking ingress or egress traffic to or from a virtual machine.
https://docs.microsoft.com/en-us/azure/network-watcher/network-watcher-monitoring-overview
https://docs.microsoft.com/en-us/azure/network-watcher/network-watcher-ip-flow-verify-overview
Azure Traffic Analytics:
Traffic Analytics is a cloud-based solution that provides visibility into user and application activity in cloud networks. Traffic analytics analyses Network Watcher network security group ( NSG ) flow logs to provide insights into traffic flow in your Azure Cloud. With traffic analytics, you can:
Identity security threats to and secure your network with information such as open ports, application attempting internet access, and virtual machines ( VM ) connecting to rogue networks.
Visualize network activity across your Azure Subscription and identify hot spots.
Understand traffic flow patterns across Azure regions and the internet to optimize your network deployment for performance and capacity.
Pinpoint network misconfigurations leading to failed connections in your network.
Azure Service Map:
Service Map automatically discover application components on Windows and Linux Systems and maps the communication between services. With Service Map, you can view your servers in the way that you think of them: as interconnected systems that deliver critical services. Service Map shows connections between servers, processes, inbound and outbound connection latency, and ports across any TCP - connected architecture, with no configuration required other than the installation of any agent.
https://docs.microsoft.com/en-us/azure/network-watcher/traffic-analytics
https://docs.microsoft.com/en-us/azure/azure-monitor/insights/service-map
There are two identity provider :
1) Synchronized identity : User management occurs on-premises. Azure AD authenticates employees by using on-prem passwords.
Azure AD Domain Services for hybrid organization
Organizations with a hybrid IT infrastructure consume a mix of cloud resources and on-premises resources. Such organisations synchronize identity information from their on-premises directory to their Azure AD tenant. As hybrid organisations look to migrate more of their on-premises applications to the cloud, especially legacy directory-aware applications, Azure AD Domain Services can be useful to them.
example: kitware corporation has deployed Azure AD Connect, to synchronize identity information from their on-premises directory to their Azure AD tenant. The identity information that is synchronized includes user accounts, their credential hashes for authentication ( password hash sync ) and group memberships.
User accounts, group memberships and credentials from kitware on-premises directory are synchronized to Azure AD via Azure AD Connect. These user accounts, group memberships and credentials are automatically available within managed domain.
2) Federated identity: User management occurs on-premises. The on-premises domain controller authenticates employee credentials.
You can federate your on-premises environment with Azure AD and use this federation for authentication and authorization. This sign-in method ensures that all user authentication occur on-premises.
Azure Monitor :
Metrics in Azure Monitor are stored in a time-series database which is optimized for analyzing time-stamped data. This makes metrics particularly suited for alerting and fast detection of issues.
Azure Log Analytics:
Log data collected by Azure monitor is stored in a log analytics workspace. which is based on Azure Data Explorer. Logs in Azure Monitor are especially useful for performing complex analysis across data from a variety of sources.
https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-platform
https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-platform-logs
Azure Network Security Group Analysis.
Network traffic for the solution must be securely distributed by providing the following features:
HTTPS protocol
Round robin routing
SSL offloading
Azure Application Gateway
if you are looking for Transport Layer Security ( TLS ) protocol ( "SSL offload") or per HTTP/HTTPS request, application-layer processing, review application gateway.
Application Gateway is a layer 7 load balancer, which means it works only with web traffic ( HTTP, HTTPS, WebSocket, and HTTP/2). It supports capabilities such as SSL termination, cookie-based session affinity, and round robin for load-balancing traffic.
Load-balances traffic layer 4 ( TCP or UDP )
https://docs.microsoft.com/en-us/azure/application-gateway/application-gateway-faq
https://docs.microsoft.com/en-us/azure/azure-monitor/log-query/log-query-overview
https://docs.microsoft.com/en-us/azure/app-service/web-sites-integrate-with-vnet
how to transform your API so it does not reveal info about the private backend. For example, you might want to hide the info about the technology stack that is running on the backend. You might also want to hide original URLs that appear in the body of the API's HTTP response and instead redirect them to the APIM gateway.
https://docs.microsoft.com/en-us/azure/azure-monitor/platform/itsmc-overview
https://docs.microsoft.com/en-us/azure/storsimple/storsimple-ova-overview
https://docs.microsoft.com/en-us/azure/virtual-machines/windows/hybrid-use-benefit-licensing
https://azure.microsoft.com/en-us/pricing/reserved-vm-instances/
What is hybrid-use-benefit-licensing ?
For customers with software assurance, azure hybrid benefit for windows server allows you to use your on-premises windows server licenses and run windows virtual machines on azure at a reduced cost. You can use azure hybrid benefit for windows server to deploy new virtual machines with windows OS.
What is Azure reserved instances ?
With Azure Reserved VM instances ( RIs ) you reserve virtual machines in advance and save upto 80 percent.
