python

Python is a general purpose high level programming language. Python was developed by Guido Van Rossam in 1989 while working at National Research Institute at Netherlands.

But officially python was made available to public in 1991. The official date of Birth for python o feb 20th 1991.

Python is recommended as first programming language for beginners.

Example : To print hello world 

Java 

2. public class HelloWorld
3. {
4.  p s v main(String[] args)
5. {
6.     SOP(“Hello world”);
7.  }
8. }

C:

#include <stdio.h>

void main()

{

   print(“Hello world”);

}

Python:

print(“Hello World”)

Eg: To print the sum of 2 numbers 

Java: 

public class Add

{

    public static void main(String[] args)

    {

         int a,b;

         a =10;

         b =20;

         System.out.println(“The Sum:” +(a+b));

          }

  }

C:

#include <stdio.h>

void main()

{

    int a,b;

    a =10;

    b =20;

    printf(“The Sum:%d”,(a+b));

 

}

Python:

a=10;

b=20;

Print(“The Sum:”,(a+b))

The name python was selected from the TV show

“The complete Monty python’s Circus”, which was broadcasted in BBC from 1969 to 1974.

Guido developed python language by taking almost all programming features from different languages.

Functional programming features from C

Object oriented programming features from C++

Scripting language features from perl and shell script 

Modular programming features from Modula-3 

Most of syntax is python derived from C and ABC languages.

In usage

Common application areas are :

Developing desktop applications 

Developing web applications 

Developing database applications 

Network programming 

Developing games 

Data analysis applications 

Machine learning 

Developing artificial intelligence applications 

IOT 

Features of Python:

1. Simple and easy to learn : python is a simple programming language. When we read python program, we can feel like reading English statements. The syntaxes are very simple and only 30 + keywords are available. When compared with other languages, we can write programs with very less number of lines. Hence more readability and simplicity. We can reduce development and cost of the project.

    

      2. Freeware and open source : we can use python software without any license and it is freeware. Its source code is open, so that we can customize based on your requirements. ( example : Jython is customized version of python to work with java application.

      3. High level programming language : python is high level programming language and hence it is programmer friendly language. Being a programmer  we are not required to concentrate low level activities like memory management and security etc.

     4. Platform independent : once we write a python program, it can run on any platform without rewriting once again. Internally PVM is responsible to convert into machine understandable form.

     5. Portability :  python programs are portable. Ie we can migrate from one platform to another platform very easily. Python programs will provide same results on any platform.

     6. Dynamically typed : In python we are not required to declare type for variables. Whenever we are assigning the value, based on value, type will be allocated automatically. Hence python is considered as dynamically typed language. But Java, C etc are statically typed language b’z we have to provide type at the beginning only.

7.  Both procedure oriented and object oriented: python language supports both procedure oriented  ( like C, pascal etc ) and object oriented ( like C++, java ) features. Hence we can get benefits of both like security and reusability etc.

      8. Interpreted : we are not required to compile python programs explicitly. Internally python interpreter will take care that compilation. If compilation fails interpreter raised syntax errors. Once complilation success then PVM ( python virtual machine ) is responsible to execute.

 

     9. Extensible : we can use other languages programs in python. The main advantages of this approach are: we can use already existing legacy non-python code. We can improve performance of the application.

     10. Embedded : we can use python programs in any other languages programs. We can embedd python programs anywhere.

      11. Extensive library : python has a rich inbuilt library. Being a programmer we can use this library directly and we are not responsible to implement the functionality etc 

Limitations of python : 

Performance wise not up to mark because it is intrepreted language. Not using for mobile applications.

Flavours of python : 

1. CPython : it is the standard flavor of python. It can be used to work with C language applications.
2. Jython or JPython : it is for Java applications. It can run on JVM 
3. IronPython : it is for C#.Net platform 
4. PyPy : The main advantage of PyPy is performance will be improved because JIT compiler is available inside PVM.
5. RubyPython : For Ruby platforms.
6. Anaconda python : it is specially designed for handling large volume of data processing.

Python versions : 

Python 1.0 v introduced in jan 1994

Python 2.0 v introduced in October 2000

Python 3.0v introduced in December 2008

Note : python 3 won’t provide backward compatibility to python2 i.e there is no guarantee that python2 programs will run in python3.

what is CWE ?

Fullform of CWE is Common Weakness enumeration 

The Common Weakness Enumeration (CWE™) Top 25 Most Dangerous Software Errors (CWE Top 25) is a demonstrative list of the most widespread and critical weaknesses that can lead to serious vulnerabilities in software. These weaknesses are often easy to find and exploit. They are dangerous because they will frequently allow adversaries to completely take over execution of software, steal data, or prevent the software from working. The CWE Top 25 is a community resource that can be used by software developers, software testers, software customers, software project managers, security researchers, and educators to provide insight into some of the most prevalent security threats in the software industry.

To create the list, the CWE Team used a data-driven approach that leverages published Common Vulnerabilities and Exposures (CVE®) data and related CWE mappings found within the National Institute of Standards and Technology (NIST) National Vulnerability Database (NVD), as well as the Common Vulnerability Scoring System (CVSS) scores associated with each of the CVEs. A scoring formula was then applied to determine the level of prevalence and danger each weakness presents. This data-driven approach can be used as a repeatable, scripted process to generate a CWE Top 25 list on a regular basis with minimal effort.

reference:

Top 25 is list here :

https://cwe.mitre.org/top25/archive/2019/2019_cwe_top25.html

reference:

one-way hash :  A one-way hash maps data of arbitrary size to a bit-string of fixed size ( the hash value ) such that the process is infeasible to invert. Used in cryptographic functions, such as strong passwords securely

salt : A salt is randow data that is used as an additional input to a one-way function that hashes a password or passpharse. it ensures the randomness of the hashed outcome, making the hash very difficult to crack.

brute-force attacks : In a brute-force attacks, the attacker iteratively tries hashing all possible values and compares the hashes to the stored hashes.
( if weak hashing is used, even strong passwords can be easily compromised by brute-force attacks).
[ In practice, even when strong hashing is used, hashed passwords might be vulnerable to dictionary attacks ].

what is dictionary attacks ?
In a dictionary attacks, the attacker hashes a list ( dictionary ) of possible passwords and compares them to stored hashes.

How to prevent weak hashing ?

Generate a random salt

[ each time you process a new password. Add the salt to the plaintext password before hashing it. When you store the hash, also store the salt ]

suggestion:
Do not use the same salt for every password that you process (CWE-760).

Use one-way hashing

[  that allow you to configure many rounds, such as bcrypt. This may increase the expense when processing incoming authentication requests, but if the hashed passwords are ever stolen, it significantly increases the effort for conducting a brute force attack, including rainbow tables

Testing [ How to test the weak hashing vulnerabilities ? ]

Find the hashes used by the application.
Determine the algorithm used for hashing.
Decide whether the hashing algorithm matches application security requirements:
Make sure that an iterative hashing algorithm with a sufficient number of rounds is used.
Make sure that a unique salt is used for each password and is stored with the hash.
Not using a salt, not using an iterative algorithm, or not using a sufficiently large amount of round all constitute weaknesses of this type.

how to setup spinnaker and jenkins using the helm provider

Step1 : 

•Login for Azure account :   az login

Step2:  To the set the account to the specific subscription \

az account set -s   ******************

Step3: To get the kubeconfig

az aks get-credentials --resource-group myResourceGroup --name myAKSCluster

Step4: Open the K8s dashboard using AZ-CLI

az aks get-credentials --resource-group myResourceGroup --name myAKSCluster

•Open the K8s dashboard using AZ-CLI

        az aks browse --resource-group demo_cluster --name demo-cluster

Step5: •Create clusterrolebinding to give access to kubernetes-dashboard(ServiceAccount) for your respective namespaces

Kubectl create clusterrolebinding view-dashboard  —clusterrole=cluster-admin  —serviceaccount=kube-system:kubernetes-dashboard

Step6:

kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default

   a.    Create ServiceAccnt & Clusterolebinding with Cluster-admin access the serviceaccnt

   apiVersion: v1

kind: ServiceAccount

metadata:

  name: tiller

  namespace: kube-system

---

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

  name: tiller

roleRef:

  apiGroup: rbac.authorization.k8s.io

  kind: ClusterRole

  name: cluster-admin

subjects:

  - kind: ServiceAccount

    name: tiller

    namespace: kube-system

Reference: https://docs.microsoft.com/en-us/azure/aks/kubernetes-helm

Step7:  helm install & init 

•helm init

Install Tiller --> helm init --service-account tiller

Extras :

Create a spinnaker :

Why this helm init --upgrade --service-account tiller ?

Error: release default failed: namespaces "default" is forbidden: User "system:serviceaccount:kube-system:default" cannot get resource "namespaces" in API group "" in the namespace "default"

That is why we need to upgrade it.

helm install stable/spinnaker --name spinnaker-demo -n default

Step8: 

•Install Traefik in Jenkins Namespace

               helm install stable/traefik --name traefik --set dashboard.enabled=true,serviceType=LoadBalancer,dashboard.domain=example-jenkinsinstance.eastus.***********.com,rbac.enabled=true  --namespace jenkins

•Once the FrontEndIP is created under Loadbalancers for Traefik in Azure portal, then create a DNS Label(same as which was specified in  the above command) for this FrontEndIP.

      Ex: jenkinsinstance.eastus.************.com

Step9:

•Create namespace called  Spinnaker and Install Spinnaker using Helm

      helm install --name spinnaker stable/spinnaker --timeout 600 --namespace=spinnaker

How to create the Dockerfile for Jsonnet

We can use this in the windows operating system.

FROM bitnami/jsonnet:latest

COPY ./demo /var

WORKDIR /var/demo/

ENTRYPOINT [ "jsonnet", "/demo-app-gce.jsonnet" ]

The Directory used in the Dockerfile  ( 

reference repo : https://github.com/spinnaker/spinnaker/tree/master/sponnet/demo

).

run this command to test the jsonnet file:

docker run -it --rm jsonnet-image

Docker container health check

Docker container health check 

For example one webhook is  running as a container

( how to know whether it is healthy or unhealthy ).

How the health check will make an added advantage to our application ?

In some scenarios, the container will be in running state, but we don’t see any interaction between the application container and the client.

One of the reason for these kind of behavior may be load.

1. In the above scenario we need to troubleshoot to understand the behavior of the container.

( we can troubleshoot the container with several approaches , for example,

docker ps 

docker ps -a  ( analyst why container is in stopped state and go through that specific container logs ).

( but in stopped state container will not provide the logs , then what is the best way to troubleshoot ??? )

How to check the healthcheck  of the container using the healthcheck command in the Dockerfile ?

What is healthcheck ?

Healthcheck are exactly  what they sound like - a way of checking the health of some resource. In the case of Docker, a health check is a command used to determine the health of a running container 

When a healthcheck command specified, it tells Docker how to test the container to see if it’s working. With no health check specified, docker has no way of knowing whether or not the services running within your container are actually up or not

Take an example  of python was flask framework 

pythonapp/Dockerfile 

FROM python:2.7

MAINTAINER Madhu Sudhan reddy  "jmstechhome@gmail.com"

COPY . /app

WORKDIR /app

RUN pip install -r requirements.txt

ENTRYPOINT ["python"]

CMD ["app.py”]

Steps 

#Simple Python Helloworld app using docker

Build the image using the following command

docker build -t pythonapp:v1 .

Run the Docker container using the command shown below.

docker run -it -p 80:5000 --name myapp pythonapp:v1

The application will be accessible at

http://<host_ip>:80

pythonapp/app.py 

from flask import Flask

app = Flask(__name__)

@app.route("/")

def hello():

       return "Hello world!!"

if __name__ == "__main__":

    app.run(debug=True,host='0.0.0.0')

pythonapp/requirements.txt

flask

>>>>

Lets start with the requirements.txt:

Flask==0.12.2

And the Dockerfile 

FROM python:3.6-alpine 

COPY . /app

WORKDIR /app

RUN pip install -r requirements.txt

CMD [“python”, “app.py”]

And finally.app.py;

from flask import Flask 

app = Flask(_name_)

@app.route(‘/‘)

def hello_world():

    return ‘Hello world’

if_name_==‘_main_’:

  app.run(host=“0.0.0.0”)

Now lets build the container 

docker  build -t docker-flask

This should build pretty quickly, Then we will run the container 

docker run —rm —name docker-flask -p 5000:5000 docker-flask

Now test by opening up your browser to localhost:5000 you should see “Hello world”

Add a health check to the Dockerfile 

Since the goal of the container is to serve the traffic on port 5000. Our health check should make sure this is happening 

A health check is configured In the Dockerfile using the HEALTHCHECK instruction. There are two ways to use the HEALTHCHECK Instruction 

HEALTHCHECK  ( OPTIONS ) CMD command 

Or if you want to disable a health check from a parent image:

HEALTHCHECKNONE

So we are obviously going to use the first. So lets add the HEALTHCHECK instruction , and we will use curl to ensure that our app is serving traffic on port 5000

So add this line to the Dockerfile right before the last line (CMD)

HEALTHCHECK CMD curl —fail http://localhost:5000/ || exit 1 

In this case, we are using the default options, which are interval 30s, timeouts 30s, start-period 0s and retries 3. Read the health check instruction reference for more information on the options.

FROM python:3.6-alpine

COPY . /app

WORKDIR /app 

RUN  app add curl 

RUN pip install -r requirements.txt

HEALTHCHECK CMD curl —fail http://localhost:5000/ || exit 1 

CMD [“python”, “app.py” ]

See the health status 

Lets rebuild and run our container 

docker build -t docker-flask.

docker run  —rm  —name docker-flask -p 5000:5000 docker-flask 

Now lets take a look at the health status. Notice we have the —name option to the above command so we can early inspect t the container

docker inspect —format=‘{{Json.State.Health}}’ docker-flask 

If you run the immediately after the container starts, you will see status is starting 

{“Status”:”starting”,”FailingStreak” :0, “Log” :[]}

And after that the health check name ( after the default interval of 30s):

{“Status”:”starting”,”FailingStreak” :0, “Log” :[ ( “start”: “27-10-23773ry23ry3ry82r3yr8”, “End”:” 2017-07-29”}

How to install Gradle ?

Install on the MacOs

The current Gradle release is 5.6. You can download binaries and view docs for all Gradle versions from the

Prerequisites 

$ java -version
java version "1.8.0_121"

Homebrew 

$ brew install gradle



Step 3. Configure your system environment



$ export PATH=$PATH:/opt/gradle/gradle-5.6/bin



Step 4. Verify your installation

$ gradle -v



Install the gradlew 



gradle wrapper --gradle-version 2.13
Starting a Gradle Daemon (subsequent builds will be faster)



Upgrade with the Gradle Wrapper



$ ./gradlew wrapper --gradle-version=5.6 --distribution-type=bin






$ ./gradlew tasks
Downloading https://services.gradle.org/distributions/gradle-5.6-bin.zip
...

Chaos engineering and Just in time access for the network, vms and the keys

Chaos Engineering 

Discipline of experimenting on a distributed system in order to build  confidence  in the system’s capability to withstand turbulent conditions in production 

Introduction:

Traditional software testing approaches like unit, regression and  integration testing validate known conditional and scenarios 

A distributed system has services whose interactions can cause unpredictable behavior in production 

To gauge stability of more complex distributed systems involving  multiple components and services and the interactions therein

Examples of  interactions 

When a service is unavailable, does it fall back  or fail gracefully without impacting the whole system E.g. can the account summary still be accessed when the payment  system is down 

What happens when things fail and retries cause additional burden on the system 

When the site is slow, what If the users keep clicking again and again. Are the transactions is idempotent or is the backend overloaded 

Ramfications:

Outages can have negative impact on brand reputation 

Engineering costs to retroactively figure out the root cause of an issue 

SLA breach can lead to service providers having to compensate 

Need for chaos engineering 

To comprehend the systematic effect of changes in distributed system 

Understand vulnerable points of service 

Improve resiliency of the system 

Advantages 

Customer satisfication 

Increased availability  and durability means no business outages for 

More technical insights of the services:

Better understanding of system failure modes 

Improved mean time to detection for issues 

Reduction in repeated issues 

Chaos engineering Stages 

Stage0 : 

Preparing for disaster 

Establish observability 

Define the critical dependencies 

Define the non-critical dependencies 

Create a disaster recovery failover playbook 

Create a critical dependency failover playbook 

Create a non-critical dependency failover playbook 

Publish the above and get team-wide agreement 

Manually execute a failover exercise 

Stage01 

Injecting chaos internally 

Perform critical dependency failure tests in non-production 

Publish test results 

( like a vaccine, we inject harm to build immunity ).

Stage02 

Pushing the envelope forward 

Perform frequent, semi-automated tests 

Execute a resiliency experiment in prod 

Publish test results 

Stage03

Automating chaos internally 

Automate resiliency testing in non-production 

Semi-automate disaster recovery failover 

Stage04

Injected automated chaos everywhere 

Integrate resiliency testing in CI/CD

Automate resiliency and disaster recovery failover testing in production 

Gremlin : Injecting chaos in example services 

Types of attacks 

Shutdown 

Time travel 

CPU

Disk 

Black hole 

IO

DNS

Memory 

Latency 

Process killer 

Packet Loss 

Application-level 

GremlinD    >>>>>> Register with plane via secret based authentication   >>>>  plane 

Gremlin      >>>>>>. Attack orders from users executed by gremlin client on host machine >>>> plane 

How to implement chaos engineering ?

In example service 

Identify all the candidate components for attacks 

Create CPU and shutdown attack scripts 

Attack generation on collector and logging tier 

Integration with slack for alerting the teams 

2)

Attacks to be created for other services 

Automated attacks 

Resiliency implementation 

Controlled automated attacks in all envs - sandbox and qa 

JTAP  ( Just-in-time access to prod ) 

What’s wrong with how things are now ?

No AD integration 

Cross-cloud makes things tricky 

Revocation is messy 

Giving someone temporary access is not possible 

How JTAP works ?

A web app that users login to and request access 

User authentication via AD + MFA 

User will be provided with unique credentials that have an expiry date

We are allowing the SSH key for four hours 

SSH 

1. ssh -I  vis user@publicipaddress
2. eval $(ssh-agent -s)
3. ssh-add -t 3600 vic
4. Ssh-keygen -Lf vis-cert.pub ( it will show the validity for the 4 hours )
5. Cd /var/log  >>  auth.log 
6. tail auth.log  

Pros 

No third-party dependency 

No config change on target VMs every time there’s a new request 

Existing methods will work 

Auditing / traceability 

Can revoke access for individual users, even after cert is issued

Things to keep in mind 

Secrety of the CA key is paramount 

Regular key rotation is recommended