Deploy Springboot Microservices App into Amazon EKS Cluster using Jenkins Pipeline | Containerize Springboot App and Deploy into EKS Cluster using Jenkins Pipeline

We will learn how to automate Docker builds using Jenkins pipelines and Deploy into AWS EKS - Kubernetes Cluster with help of Kubernetes Continuous Deploy plug-in.

We will use Springboot Microservices based Java application. I have already created a repo with source code + Dockerfile. The repo also have Jenkinsfile for automating the following:

- Automating builds using Jenkins
- Automating Docker image creation
- Automating Docker image upload into Docker Hub
- Automating Deployments to Kubernetes Cluster

 

 

Please watch the above steps in YouTube channel:

Pre-requisites:
1. Amazon EKS Cluster is setup and running. Click here to learn how to create Amazon EKS cluster.

2. Jenkins Master is up and running

3. Setup Jenkins slave, install docker in it.
4. Docker, Docker pipeline and Kubernetes Continuous Deploy plug-ins are installed in Jenkins

5. Docker hub account setup in https://cloud.docker.com

6. Install kubectl on your instance

Step #1 -Make sure Jenkins can run Docker builds after validating per pre-requisites

Step #2 - Create Credentials for Docker Hub

Go to Jenkins UI, click on Credentials -->

Click on Global credentials

Click on Add Credentials

Now Create an entry for your Docker Hub account. Make sure you enter the ID as dockerhub

Step #3 - Create Credentials for Kubernetes Cluster

Click on Add Credentials, use Kubernetes configuration from drop down.

execute the below command to get kubeconfig info, copy the entire content of the file:

sudo cat ~/.kube/config

Enter ID as K8S and choose enter directly and paste the above file content and save.

Step # 4 - Create Maven3 variable under Global tool configuration in Jenkins

Make sure you create Maven3 variable under Global tool configuration.

 

 

Step #5 - set a clusterrole as cluster-admin

By default, clusterrolebinding has system:anonymous set which blocks the cluster access. Execute the following command to set a clusterrole as cluster-admin which will give you the required access.

kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous

Step # 6 - Create a pipeline in Jenkins

Create a new pipeline job.

Step # 7-  Copy the pipeline code from below
Make sure you change red highlighted values below as per your settings:
Your docker user id should be updated.
your registry credentials ID from Jenkins from step # 1 should be copied

node ("slave") {
  def image
  def mvnHome = tool 'Maven3'
     stage ('checkout') {
        checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://bitbucket.org/ananthkannan/myawesomeangularapprepo/']]])      
        }
   
    stage ('Build') {
            sh 'mvn -f MyAwesomeApp/pom.xml clean install'           
        }
       
       
    stage ('Docker Build') {
         // Build and push image with Jenkins' docker-plugin
            withDockerRegistry([credentialsId: "dockerhub", url: "https://index.docker.io/v1/"]) {
            image = docker.build("akdevopscoaching/mywebapp", "MyAwesomeApp")
            image.push()    
            }
        }

      stage ('K8S Deploy') {
       
                kubernetesDeploy(
                    configs: 'MyAwesomeApp/springboot-lb.yaml',
                    kubeconfigId: 'K8S',
                    enableConfigSubstitution: true
                    )               
        }
    
}

Step # 8 - Build the pipeline

Once you create the pipeline and changes values per your Docker user id and credentials ID, click on 

Step # 9 - Verify deployments to K8S

kubectl get pods

kubectl get deployments

kubectl get services

Steps # 10 - Access SpringBoot App in K8S cluster

Once build is successful, go to browser and enter master or worker node public ip address along with port number mentioned above

http://master_or_worker_node_public_ipaddress:port_no_from_above

You should see page like below:

Code error 403 when trying to access Kubernetes cluster | Jenkins Kubernetes Deployment Error

When ever you are doing deployment from Jenkins to EKS cluster, you may get this error:

Api call failed with code 403, detailed message: {
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },

"status": "Failure",
  "message": "namespaces is forbidden: User \"system:anonymous\" cannot list namespaces at the cluster scope",
  "reason": "Forbidden",
  "details": {
    "kind": "namespaces"
  },
  "code": 403 

Work around or the fix: 

You get this error because you're getting blocked by RBAC policies. Basically, RBAC policies set to restrict the resources you use and limits a few of your action. 

There are two possibilities, either you haven't created an RBAC or it's somehow restricting the cluster access.

By default, your clusterrolebinding has system:anonymous set which blocks the cluster access.

Execute the following command, it will set a clusterrole as cluster-admin which will give you the required access.

kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous

How to fix Jenkins Error | Incorrect Java 11 version

You may have this issue when trying to install Jenkins with default JDK version(Java 11) in Ubuntu 18.0.4. We need to fix by installing JDK 8.

Remove Jenkins first.

sudo apt-get remove jenkins -y

 

Install Java 8 version

sudo apt-get install openjdk-8-jdk -y

Now choose which version to select from both Java 11 and Java 8 

sudo update-alternatives --config java

type 2 to choose Java 8 version.

Check Java version after setting:

Install Jenkins

sudo apt-get install jenkins -y

Now try to access Jenkins in the browser.

Slack Azure DevOps Integration | How to Integrate Slack with Azure DevOps | Send push notifications from Azure pipelines to Slack

Slack is popular collaboration tool used at work by many companies. Slack can be integrated with CI tools such as Jenkins, Azure DevOps(VSTS). We will see how to integrate with Azure Build pipelines. 

Pre-requistes:

1. Azure pipelines setup

2. Slack channel setup

 

Azure DevOps & Slack integration steps

 

1. Go to slack.com and sign in to workspace.
2. Go to channel where you would like to receive push notification from Azure DevOps. Click on more and Add apps

3. Add Visual Studio Team Services App

4. Add Configuration

5.  And then select channel to post notification and add the channel

6. In Azure DevOps Dashboard, click on Project settings.

7. Click on Service Hooks tab and then click create subscriptions button to add a new service hook.
 

8. Select Slack from the list of services, and press the Next button.

9. On the Trigger screen, choose the event that you’d like to trigger the notification, and add any filters. Press the Next button to continue.

10. On the Action screen, add the URL as the Slack Webhook URL and press Finish when you are done.

 

 

Now whenever Azure pipelines runs any builds, it will send push notifications to the Slack channel. Please watch the above steps in YouTube channel:

How to Install TeamCity on AWS Ubuntu EC2 | How to setup TeamCity on Ubuntu 18.0.4 with PostgreSQL?

TeamCity is one of the CI tools, It is open source and Java based tool. It can be configured easily.

Please find below steps for installing TeamCity on Ubuntu:

Pre-requisites:
Make sure Ubuntu instance has at least 4 GB RAM
Open port 8111 in security firewall.

We need to setup database for storing all the build information. We will be using PostgreSQL.

PostgreSQL Installation

sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list'

 

sudo wget -q https://www.postgresql.org/media/keys/ACCC4CF8.asc -O - | sudo apt-key add -

sudo apt-get -y install postgresql postgresql-contrib

 

sudo systemctl start postgresql

sudo systemctl enable postgresql

Login as postgres user

sudo su - postgres

Now create a user below

createuser teamcity

Switch to sql shell by entering
psql

 

 

 

 

 

 

Execute the below three lines (one by one)

 

ALTER USER teamcity WITH ENCRYPTED password 'password';
CREATE DATABASE teamcity OWNER teamcity;
\q

type exit to come out of postgres user.

TeamCity Installation

Download Java

sudo apt-get update && sudo apt-get install default-jdk -y

Download TeamCity

wget https://download.jetbrains.com/teamcity/TeamCity-2020.1.5.tar.gz

Extract TeamCity
tar -xvf TeamCity-2020.1.5.tar.gz
sudo mkdir /opt/JetBrains
sudo mv TeamCity /opt/JetBrains/TeamCity
cd /opt/JetBrains/TeamCity
 

Create service file

sudo nano /etc/init.d/teamcity
(copy the whole text colored in green)
#!/bin/sh
### BEGIN INIT INFO
# Provides:          TeamCity autostart
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start teamcity daemon at boot time
# Description:       Enable service provided by daemon.
# /etc/init.d/teamcity -  startup script for teamcity
### END INIT INFO

#  Ensure you enter the  right  user name that  TeamCity will run  under
USER="ubuntu"

export TEAMCITY_DATA_PATH="/opt/JetBrains/TeamCity/.BuildServer"

case $1 in

start)
  start-stop-daemon --start  -c $USER --exec /opt/JetBrains/TeamCity/bin/runAll.sh start
 ;;
stop)
  start-stop-daemon --start -c $USER  --exec  /opt/JetBrains/TeamCity/bin/runAll.sh stop
 ;;
 esac

exit 0

Ctrl + O enter
Ctrl + X enter
sudo chmod +x /etc/init.d/teamcity
sudo update-rc.d teamcity defaults
sudo /etc/init.d/teamcity start

Access logs files

cat /opt/JetBrains/TeamCity/buildAgent/logs/teamcity-agent.log

Now access Teamcity UI

Open browser and access team city url in the browser
http://public_dns_name:8111

Click on Proceed.
Select database type as jdbc driver as Postgres SQL from the dropdown and Click on Proceed.

now you need to download the jdbc driver at below location.
cd /opt/JetBrains/TeamCity/.BuildServer/lib/jdbc
sudo wget https://jdbc.postgresql.org/download/postgresql-9.4.1212.jar

Select database type as PostgreSQL
Refresh JDBC driver
You should see like this —> Loaded PostgreSQL JDBC driver version: 9.4

Now provide the below info
Enter database Host - localhost
Enter database name - teamcity
User name - teamcity
Password - password
Click on Proceed

please wait and watch..As It may take a few mins…

Scroll down the page, Accept license agreement
Uncheck Send anonymous usage statistics
Continue button, Register as a user and create an account

How to Fix The following signatures couldn’t be verified Error in Ubuntu Linux | The following signatures couldn’t be verified Error in Ubuntu Linux

W: GPG error: https://pkg.jenkins.io/debian-stable binary/ Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY FCEF32E745F2C3D5
E: The repository 'http://pkg.jenkins.io/debian-stable binary/ Release' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.

Fix GPG error: The following signatures couldn’t be verified

What we need to do is to fetch this public key in the system. Get the key number from the message. In the above message, the key is XXXXXXXXXXX. Now use that key in the following command:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys XXXXXXXXXXX

This will add the key to the system. and then perform update

 sudo apt-get update 

and you should not see this error any more. I hope this solved the problem .

 

Install Azure CLI in Ubuntu 18.0.4 | How to setup Azure CLI in Ubuntu 18.0.4 |

Azure CLI can be installed by following below steps:

Run the update first
sudo apt-get update

sudo apt-get install ca-certificates curl apt-transport-https lsb-release gnupg

curl -sL https://packages.microsoft.com/keys/microsoft.asc |
    gpg --dearmor |
    sudo tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null

AZ_REPO=$(lsb_release -cs)
echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" |
    sudo tee /etc/apt/sources.list.d/azure-cli.list

sudo apt-get update
sudo apt-get install azure-cli

Run the Azure CLI with the az command. To sign in, use the az login command.

az login

How to change default port number in Tomcat? | Change default port number in Tomcat | Change default port number in Tomcat

The default port number for Tomcat is 8080. It can be changed though by modifying the below properties file.

sudo vi /var/lib/tomcat9/conf/server.xml

 

 

 

Modify the desired port no in the above file. once you modify, you need to restart Tomcat service to take effect.

sudo service tomcat9 stop

sudo service tomcat9 start

sudo service tomcat9 status
Click here to see steps for how to install Tomcat9 on Ubuntu.

How to create AKS cluster in the Azure portal | How to connect to AKS cluster in Azure Portal

 Azure Kubernetes Service (AKS)

AKS allows you to quickly deploy a production ready Kubernetes cluster in Azure, deploy and manage containerized applications more easily with a fully managed Kubernetes service. 

AKS reduces the complexity and operational overhead of managing Kubernetes by offloading much of that responsibility to Azure. As a hosted Kubernetes service, Azure handles critical tasks like health monitoring and maintenance for you. The Kubernetes masters are managed by Azure. You only manage and maintain the agent nodes. As a managed Kubernetes service, AKS is free - you only pay for the agent nodes within your clusters, not for the masters.

We will see how to create AKS cluster in portal Azure.

1. go to https://portal.azure.com/#home

2. Click on Kubernetes services

3. Click on Add Kubernetes Cluster

 

4. Click on Add Kubernetes Cluster

 Enter information below:

 

 

5. Enter node count as 2, choose standard size

6. Click on Create after making sure validations are passed:
 

This is will take at least 2 to 5 mins to create AKS clusters. If all goes well, you should be able to see the cluster.

Once it is created, go to Kubernetes Services, click on cluster name.

 

 

How to connect to AKS cluster using Azure CLI

Now click on the cluster name, click on connect.

Make sure you have Azure CLI is installed on your local machine in order to connect to AKS cluster.

Connect to Azure portal using Azure CLI first.

az login

Enter your Microsoft credentials

az account set --subscription subscription_id

az aks get-credentials --resource-group AKS-POC --name AKS-POC

# List all deployments in a specific namespace
kubectl get deployments --all-namespaces=true

# List all deployments in a specific namespace
kubectl get deployments --namespace kube-system

 

Please watch the above steps in YouTube video:

How to change default port number for SonarQube? | Change default port number in Sonarqube | SonarQube default port number change

The default port number for SonarQube is 9000. It can be changed though by modifying the below properties file.

$sonar_install-dir/conf/sonar.properties

 

 

Let's say you want to change from default port 9000 to 9095. you need to make changes in the above file and restart the service.

once you modify, you need to restart SonarQube service to take effect.

sudo systemctl stop sonar

sudo systemctl start sonar

sudo systemctl status sonar

Now you can access SonarQube in the new port 9095.

Note: Make sure you open the new port no in security firewall rules.
Please watch how to do this YouTube as well:
 

How to change default port number in Sonatype Nexus 3? | Change default port number in SonaType Nexus 3 | Change default port number in Nexus

The default port number for Sonatype Nexus is 8081. It can be changed though by modifying the below properties file.

$install-dir/etc/nexus-default.properties

Modify the desired port no in the above file. once you modify, you need to restart Nexus service to take effect.

sudo service nexus stop

sudo service nexus start

sudo service nexus status

 

Click here to see steps for how to install Nexus on Redhat.
Click here to see steps for how to install Nexus on Ubuntu.