Groovy jenkins-pipeline
Need help with cheatsheet-jenkins-groovy-A4?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.


:book: Groovy CheatSheet For Jenkins Usage In A4

172 Stars 145 Forks 73 Commits 1 Opened issues

Services available

Need anything else?

  • CheatSheet: Jenkins & Groovy :Languages: :PROPERTIES: :type: language :exportfilename: cheatsheet-jenkins-groovy-A4.pdf :END:



PRs Welcome


  • PDF Link: [[][cheatsheet-jenkins-groovy-A4.pdf]], Category: [[][languages]]
  • Blog URL:
  • Related posts: [[][Jenkins CheatSheet]], [[][#denny-cheatsheets]]

File me [[][Issues]] or star [[][this repo]]. ** Jenkins Pipeline | Name | Comment | |-------------------------------------------+---------------------------------------------------------------------------| | Specify parameter to run jobs | =build job:'job1', parameters:[string(name:'name1', value:va1)]= | | Run job in different agents | =node($agentlabel) {...}= | | Ask for user input | =stage('stage2'){ input "OK to go?" }= | | Actively fail current pipeline job | =error("Build failed because of this and that..")= | | List all Jenkins plugins and versions | [[][list-all-plugins.groovy]] | | [[][Check whether property exists]] | =if (env.keepfailedenv)= | | [[][Jenkins Pipeline enable timestamps]] | =options{timestamps()}= | | [[][Set envs within a jenkins pipeline]] | =withEnv(["key1=$var1"])= | | Install plugin via groovy | =Hudson.instance.updateCenter.getPlugin(plugin).deploy().get()= | | Keep previous job run via groovy | =buildDiscarder(logRotator(daysToKeepStr: '20', numToKeepStr: '60'))= | | [[][Customize job workspace]] | =customWorkspace "/some/other/path"= | | [[][git scm checkout to relative directory]] | =extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'XXX']]= | | Keep going when previous stage has failed | [[][keep-going-with-errors.groovy]] | | Send slack notification in pipeline | [[][slack-notification.groovy]] | | Pass parameter across jenkins jobs | [[][jenkinsfile-pass-parameter.groovy]] | | Set timeout & retry | [[][jenkinsfile-timeout-retry.groovy]] | | Use finally to do cleanup | [[][jenkinsfile-finally.groovy]] | | Run jenkins jobs in a sequential way | [[][jenkinsfile-sequentially.groovy]] | | Run jenkins jobs in parallel | [[][jenkinsfile-parallelly.groovy]] | | Reference | [[][Link: Syntax Reference]], [[][Link: Jenkins User Documentation]] | | Reference | [[][Link: Groovy Language Documentation]] | | Reference | [[][Link: Example]], [[][Link: Example]] | ** Config Jenkins Via Groovy | Name | Comment | |--------------------------------------+-------------------------------------------------------------------------------------------| | Set timezone for jenkins | [[][timezone.groovy]] | | Set timezone for jenkins | =System.setProperty('org.apache.commons.jelly.tags.fmt.timeZone', 'America/LosAngeles')= | | Configure default view | [[][jenkins-views.groovy]] | | Configure Jenkins url | [[][jenkins-url.groovy]] | | Create a Jenkins user | [[][create-jenkins-user.groovy]] | | Groovy manages files/folders | [[][files-folder.groovy]] | | Configure max executors in Jenkins | [[][master-executors.groovy]] | | Configure only 1 executor per worker | For each agent, configure # of executors | | Configure slack plugin | [[][config-slack.groovy]] | | Configure pipeline shared libraries | [[][config-pipeline-library.groovy]] | | [[][Get jenkins version from CLI]] | =java -jar /usr/share/jenkins/jenkins.war --version= | | Reference | [[][GitHub: cloudbees/jenkins-scripts]], [[][GitHub: jenkinsci/pipeline-examples]] | ** Jenkins Trouble Shooting | Name | Comment | |--------------------------------------------------+---------------------------| | List performance metrics for each Jenkins agents | [[][]] | ** Jenkins Kubernetes Via Groovy | Name | Comment | |-------------------------------------+---------------------------------------------| | Config jenkins kubernetes plugin | [[][jenkins-kubernetes-cloud.groovy]] | | Validate Kubernetes jenkins setup | [[][validate-kubernetes-cloud.groovy]] | | Kubernetes run with envs configured | [[][runWithEnvVariables.groovy]] | | Reference | [[][GitHub: kubernetes-plugin pipeline examples]] | ** Jenkins View Via Groovy | Name | Comment | |-----------------------------------------+----------------------------------------------------------------------------| | [[][Add a list of jobs by regexp to a view]] | =myView.setIncludeRegex(".Integration.")=, [[][addjobstoview-byregexp.groovy]] | | Create jenkins views and add jobs to it | [[][jenkins-views.groovy]] | | [[][Add a view of build monitor view plugin]] | [[][build-monitor-views.xml]] | | [[][Change view description in groovy]] | =myView.doSubmitDescription= |



** Jenkins Job Via Groovy | Name | Comment | |-------------------------------------------------+---------------------------------------------------------| | List all my jenkins jobs | =println Jenkins.instance.projects.collect { }= | | List all jenkins jobs | [[][list-all-jobs.groovy]] | | Create and trigger a job | [[][create-jenkins-job.groovy]] | | Manage jenkins jobs | [[][manage-jenkins-jobs.groovy]] | | Cancel queued jenkins jobs by regexp | [[][kill-queued-jenkins.groovy]] | | [[][Support HTML for job and parameter descriptions]] | [[][Link: OWASP Markup Formatter Plugin]] | ** Jenkins Different Parameters | Name | Comment | |----------+--------------------------------------------------------------------------------------| | string | =string(name: 'key1', defaultValue: 'Default value', description: 'some parameter')= | | text | =text(name: 'key1', defaultValue: 'Default value', description: 'some parameter')= | | boolean | =booleanParam(name: 'key1', defaultValue: true, description: 'some parameter')= | | choice | =choice(name: 'key1', choices: 'One\nTwo\nThree\n', description: 'some parameter')= | | password | =password(name: 'key1', defaultValue: 'SECRET', description: 'Enter a password')= | | file | =file(name: 'key1', description: 'Choose a file to upload')= | ** Jenkins Security Via Groovy | Name | Comment | |------------------------------------------+------------------------------------------------------------------| | logged-in users can do anything | [[][logged-in-users.groovy]] | | [[][Enable ldap in Jenkins]] | [[][enable-ldap.groovy]] | | Create a jenkins secret text | [[][create-secret-text.groovy]] | | Configure authorization in Jenkins | [[][matrix-authorization-strategy.groovy]] | | [[][Jenkins skip wizzard when initialization]] | -Djenkins.install.runSetupWizard=false | | [[][Jenkins skip wizzard when initialization]] | =instance.setInstallState(InstallState.INITIALSETUPCOMPLETED)= | | [[][Slave To Master Access Control]] | [[][00-slave-to-master-access.groovy]] | | [[][CSRF Protection]] | [[][00-csrf.groovy]] | | Add Jenkins permission | [[][jenkins-permission.groovy]] | | Disable CLI over Remoting | [[][00-disable-cli-remoting.groovy]] | | Disable jnlp | =jenkins.setSlaveAgentPort(-1)= | | [[][Access Control for Builds]] | [[][]] | ** Load Jenkins settings via folder copy | Name | Comment | |--------------------------------------+--------------------------------------------------------------------------| | [[][Add default jobs]] | =Copy jobs/ /usr/share/jenkins/ref/jobs/= | | Copy custom built plugins | =COPY plugins/.hpi /usr/share/jenkins/ref/plugins/= | | Use jenkins cli | =COPY config/ /usr/share/jenkins/ref/= | | Add jenkins groovy scripts | =COPY config/.groovy /usr/share/jenkins/ref/init.groovy.d/= | | Configure Jenkins with some defaults | =COPY config/*.xml /usr/share/jenkins/ref/= | | [[][Install jenkins plugins]] | =/usr/local/bin/ < /usr/share/jenkins/ref/plugins.txt= |



** Jenkins Plugins | Plugin | Summary | |----------------------------+---------------------------------------------------------------------------| | [[][Kubernetes Plugin]] | Jenkins plugin to run dynamic agents in a Kubernetes/Docker environment | | [[][Credentials Plugin]] | Load the ssh key | | [[][SiteMonitor Plugin]] | Monitor URLs | | [[][Timestamper Plugin]] | Add timestamp to job output | | [[][Dashboard View Plugin]] | Create dashboard | | [[][Log Parser Plugin]] | Parse the console output and highlight error/warning/info lines. | | [[][Build-timeout Plugin]] | Abort if job takes too long | | [[][Naginator Plugin]] | Retry failed a job | | [[][ThinBackup Plugin]] | Backup jenkins | | [[][JobConfigHistory Plugin]] | Backup job configuration | | [[]["Anything Goes" formatter]] | use JavaScript inside your project description | | [[][AnsiColor Plugin]] | Add support for ANSI escape sequences, including color, to Console Output | | [[][Build User Vars Plugin]] | Describe the user who started the build | | [[][Gitlab Plugin]] | Allows GitLab to trigger Jenkins builds | | [[][Workspace Cleanup]] | Plugin to delete the build workspace. | | [[][UpdateSites Manager plugin]] | manage update sites, where Jenkins accesses in order to retrieve plugins | ** Jenkins Git Via Groovy | Name | Comment | |----------------------------------------+-----------------------------------| | Git checkout code | [[][git-checkout.groovy]] | | Get all git commits since last success | [[][git-commits-before-fail.groovy]] | | List git tags and branches | [[][git-list-tags-and-branches.groovy]] |

** Jenkins networking Via Groovy | Name | Comment | |---------------------------------+--------------------------------------------------------------------------| | Get hostname | =println InetAddress.localHost.canonicalHostName= | | Get IP address | =println InetAddress.localHost.hostAddress= | | Get hostname by ip | [[][get-ip-by-hostname.groovy]] | | validate user input: ip address | =assert ipaddress.matches("\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}")= | ** Jenkins with Kubernetes/Docker | Name | Comment | |----------------------------------------------------+-------------------------------------------------------------------------| | [[][Kubernetes Plugin]] | Jenkins plugin to run dynamic agents in a Kubernetes/Docker environment | | Config jenkins kubernetes plugin | [[][jenkins-kubernetes-cloud.groovy]] | | Cleanup for Docker stale containers/images/volumes | [[][docker-cleanup.groovy]] | | Jenkins customize docker build args | =additionalBuildArgs '--build-arg SSHPRIVATE_KEY...'= | ** Groovy Common Errors/Exceptions | Name | Comment | |--------------------+--------------------------------------------| | Illegal class name | [[][JVM doesn't like class names with a hyphen]] |

** Groovy Basic | Name | Comment | |--------------------------------+-----------------------------------------------------------| | Get environment variables | [[][get-env.groovy]], =println env.WORKSPACE= | | Groovy execute command | [[][execute-command.groovy]] | | [[][Get data type of a variable]] | =myObject.getClass()= | | Print stdout | [[][print.groovy]] echo 'Action is done', println "Hello World" | | Use boolean parameter | if (istrue == "false") {...} | | Basic integer caculation | def a = 3, b = 7; println "$a + $b = ${a + b}" | | Run groovy online | [[][SaaS: Groovy Web console]] | | Run groovy script from Jenkins | [[][Link: Jenkins Script Console]] | | Reference | [[][Link: Apache Groovy]] |



** Groovy String | Name | Comment | |--------------------------------+----------------------------------------------------------------| | Check string startsWith | =assert s.startsWith("\t")= | | Trim whitespaces | s=s.trim() | | Concat string | =first = 'Joe'; last = 'Smith'; println("Name: $first $last")= | | [[][Convert list to string]] | =l.join(";")= | | Create string with multi-lines | [[][multi-line-string.groovy]] | | Convert string to list | [[][split-string.groovy]] | | [[][Convert string to json]] | [[][string-to-json.groovy]] | | Remove tags | =input.replaceAll("\<.?>","")= | | Regex match | [[][regexp-match.groovy]] | * Groovy Regexp | Name | Comment | |------------------------+--------------------------------| | [[][Regex case insensitive]] | ( == ~/(?i).NSX./ ) | | Reference | [[][Regular Expressions in Groovy]] | ** Groovy Array | Name | Comment | |-----------------------------+------------------------------------------| | Iterate a list | =for(item in [1,2,3,4]){ println item }= | | Iterate a list | =(1..3).each { println "Number ${it}"}= | | Add item to list | =def alist = [10, 9, 8]; alist << 7= | | List size | =def alist = [10, 9, 8]; alist.size()= | | Split string with delimiter | ='1128-2'.tokenize('-')= | ** Groovy File | Name | Comment | |---------------------------------+-------------------------------------------------------------------------| | [[][Read file into a string]] | =String fileContents = new File('/tmp/test.txt).text= | | Read file content as a variable | =def env = System.getenv()=, =def content = readFile("/tmp/test.txt")= | | [[][Write file in pipeline]] | writeFile file: "output/my.txt", text: "This is a test" | | [[][Read a property file]] | def conf = readProperties file: "${env.WORKSPACE}@script/" | | Read and write json files | [[][json-file.groovy]] | | Obtain a relative path | [[][relative-path.groovy]] | ** Groovy Shell Command | Name | Comment | |-----------------------------+-----------------------------------------------------| | Run shell and get output | def out = sh script: command, returnStdout: true | | Run shell and get exit code | def status = sh script: command, returnStatus: true | ** Groovy Dictionary | Name | Comment | |---------------------+------------------------------------------------| | Create a map | =def m = ['fruit':'Apple', 'veggie':'Carrot']= | | Add an item to map | =m.put('denny','hello')= | | Check if key exists | =m.containsKey('key1')= | | Loop a map | [[][loop-map.groovy]] | ** Groovy json | Name | Comment | |----------------------------+-----------------------| | [[][Convert string to json]] | [[][string-to-json.groovy]] | | Convert dictionary to json | [[][dict-to-json.groovy]] | | Read and write json files | [[][json-file.groovy]] |



** Groovy Date | Name | Comment | |----------------+------------------------------------------------------------------------------| | Date to string | =new Date().format("yyyy-MM-dd'T'HH:mm:ss'Z'", TimeZone.getTimeZone("UTC"))= | | String to date | =Date.parse("yyyy-MM-dd'T'HH:mm:ss'Z'", "2001-01-01T00:00:00Z")= | | String to date | =Date.parse("yyyy-MM-dd'T'HH:mm:ssZ", "2001-01-01T00:00:00+0000")= | ** Jenkins Agent | Name | Comment | |-------------------------------------+--------------------------------| | Check jenkins slave jar version | [[][check-slave-jar-version.groovy]] | | Find dead executors and remove them | [[][find-dead-executors.groovy]] | | Set env for each agent | [[][set-agent-env.groovy]] | ** Jenkins Maintenance | Name | Comment | |---------------------------------+---------------------------------------------------------| | Delete jenkins job by regexp | [[][delete-job-by-regexp.groovy]] | | Deploy Jenkins via docker | | | Clean up old builds | [[][Link: CloudBees Best Strategy for Disk Space Management]] | ** More Resources

License: Code is licensed under [[][MIT License]].


linkedin github slack


  • org-mode configuration :noexport: #+STARTUP: overview customtime noalign logdone showall #+DESCRIPTION: #+KEYWORDS: #+LATEXHEADER: \usepackage[margin=0.6in]{geometry} #+LaTeXCLASSOPTIONS: [8pt] #+LATEXHEADER: \usepackage[english]{babel} #+LATEXHEADER: \usepackage{lastpage} #+LATEXHEADER: \usepackage{fancyhdr} #+LATEXHEADER: \pagestyle{fancy} #+LATEXHEADER: \fancyhf{} #+LATEXHEADER: \rhead{Updated: \today} #+LATEXHEADER: \rfoot{\thepage\ of \pageref{LastPage}} #+LATEXHEADER: \lfoot{\href{}{GitHub:}} #+LATEXHEADER: \lhead{\href{}{Blog URL:}} #+AUTHOR: Denny Zhang #+EMAIL: [email protected] #+TAGS: noexport(n) #+PRIORITIES: A D C #+OPTIONS: H:3 num:t toc:nil \n:nil @:t ::t |:t ^:t -:t f:t *:t <:t #+OPTIONS: TeX:t LaTeX:nil skip:nil d:nil todo:t pri:nil tags:not-in-toc #+EXPORTEXCLUDETAGS: exclude noexport #+SEQTODO: TODO HALF ASSIGN | DONE BYPASS DELEGATE CANCELED DEFERRED #+LINKUP: #+LINK_HOME:
  • # --8<-------------------------- separator ------------------------>8-- :noexport:
  • TODO groovy challenges :noexport: ** 101 Use groovy to add a test user in Jenkins For better security, use groovy to only allow registered user login Quiz:

Once I have enabled Jenkins security, how my chef update will work? ** 102 Define a Jenkins pipeline job automatically Define a Jenkins job via Jenkinsfile script automatically For automated backup, enable and configure ThinBackup plugin via Groovy ** 301 Define a dummy Jenkins pipeline job using Jenkinsfile Define a Jenkins parameterized pipeline job using Jenkinsfile. It shall trigger another job. Quiz:

Once I have enabled Jenkins security, how my chef update will work? * TODO [#A] Blog: Jenkins pipeline: run multiple related jobs in a managed order :noexport:IMPORTANT: ** basic use Jenkins Pipeline is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins. ** TODO jenkins pipeline show slack error message ** TODO why unecessary delay when running jobs via pipeline: ** TODO Why pipeline scheduling takes serveral minutes ** TODO [#A] How to support testing different branch with scm: ** TODO [#A] Jenkins pipeline doesn't set who initiate the deployment jenkins APP [11:32 AM] RefreshDemoEnvAll - #7 Started by upstream project "PipelineRefreshDemoEnvAll" build number 3 (Open) ** TODO [#A] kill in jenkins job doesn't stop the bash: curl [email protected]:~/elasticsearch-cli-tool# curl "http://${esip}:9200/alias/staging-8a18aa800e5911e785f24a8136534b63" {"staging-index-8a18aa800e5911e785f24a8136534b63-new3":{"aliases":{"staging-8a18aa800e5911e785f24a8136534b63":{}}}}[email protected]:~/elasticsearch-cli-tool# curl "http://${esip}:9200/alias/staging-8a18aa800e5911e785f24a8136534b63" {"staging-index-8a18aa800e5911e785f24a8136534b63-new3":{"aliases":{"staging-8a18aa800e5911e785f24a8136534b63":{}}}}[email protected]:~/elasticsearch-cli-tool# ps -ef | grep curl root 11085 11062 0 22:51 ? 00:00:00 curl -XPOST -d { "conflicts": "proceed", "source": { "index": "master-index-8a18aa800e5911e785f24a8136534b63-new2", "size": "500" }, "dest": { "index": "master-index-8a18aa800e5911e785f24a8136534b63-new3", "optype": "create" } } root 11109 9468 0 22:51 pts/2 00:00:00 grep --color=auto curl root 13367 13348 0 13:05 ? 00:00:04 curl -XPOST -d { "conflicts": "proceed", "source": { "index": "master-index-321bb9606b2111e7b579a2f42be00f79-new2", "size": "500" }, "dest": { "index": "master-index-321bb9606b2111e7b579a2f42be00f79-new3", "optype": "create" } } [email protected]:~/elasticsearch-cli-tool# date ** # --8<-------------------------- separator ------------------------>8-- :noexport: ** TODO [#A] Jenkinsfile/Pipeline :IMPORTANT:

Pipeline supports two syntaxes, Declarative (introduced in Pipeline 2.5) and Scripted Pipeline *** Pipeline Vocabulary: Steps, Nodes, and Stages - A step, also known as a "build step", is a single task that we want Jenkins to execute.

  • A "node", within the contexts of a pipeline, refers to a step that does two things.

First, it schedules the defined steps so that it'll run as soon as an executor is available. Second, it creates a temporary workspace which is removed once all steps have completed.

  • And lastly, we have "Stages". Stages are for setting up logical divisions within pipelines. The Jenkins Pipeline visualization plugin will display each stage as a separate segment. Because of this, teams tend to name stages for each phase of the development process, such as "Dev, Test, Stage, and Production". *** Jenkins pipeline is durable from Jenkins master restart #+BEGIN_EXAMPLE
  • One huge benefit of using a pipeline is that the job itself is durable. A Pipeline job is able to survive planned or even unplanned restarts of the Jenkins master. If you need to survive slave failures as well, you'll have to use checkpoints.

  • Unfortunately, the checkpoints plugin is only available for the enterprise edition of Jenkins. Pipelines are also pausable.


    *** hello world: http://localhost:18083/job/jenkinsfile1/1/console ** TODO Jenkins pipeline: Supporting APIs v2.10


    INFO: Listed all plugins Nov 26, 2017 5:03:17 PM jenkins.InitReactorRunner$1 onTaskFailed SEVERE: Failed Loading plugin Pipeline: Nodes and Processes v2.8 (workflow-durable-task-step) Pipeline: Nodes and Processes v2.8 failed to load.

    • Pipeline: Supporting APIs v2.10 is older than required. To fix, install v2.12 or later. at hudson.PluginWrapper.resolvePluginDependencies( at hudson.PluginManager$2$1$ at org.jvnet.hudson.reactor.TaskGraphBuilder$ at org.jvnet.hudson.reactor.Reactor.runTask( at jenkins.model.Jenkins$7.runTask( at org.jvnet.hudson.reactor.Reactor$ at org.jvnet.hudson.reactor.Reactor$ at java.util.concurrent.ThreadPoolExecutor.runWorker( at java.util.concurrent.ThreadPoolExecutor$ at #+END_EXAMPLE ** TODO jenkins pipeline agent

The pipeline contains the entire definition that Jenkins will evaluate. Inside, we have an agent section that specifies where the actions in the pipeline will execute. To isolate our environments from the host system, we will be testing in Docker containers, specified by the docker agent.

#+BEGIN_EXAMPLE #!/usr/bin/env groovy

pipeline {

 agent {
     docker {
         image 'node'
         args '-u root'

stages { stage('Build') { steps { echo 'Building...' sh 'npm install' } } stage('Test') { steps { echo 'Testing...' sh 'npm test' } } }

} #+ENDEXAMPLE ** TODO Jenkins pipeline enable slack notificaiton ** TODO [#A] jenkins workflow: ** TODO jenkins plugins: git, checkstyle, build-pipeline-plugin, clone-workspace-scm, deploy, Text-finder ** TODO verify whether local docker image is up-to-date ** TODO Why jenkins container use so much memory? ** TODO jenkins docker demo: docker run --rm -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock --group-add=$(stat -c %g /var/run/docker.sock) jenkinsci/docker-workflow-demo ** TODO jenkins plugin: job plugin 根据前置job成功与否来执行当前job 插件链接

nice, 可以用于我们的CommonServerCheck的jenkins job依赖 ** TODO jenkins job priority ** TODO jenkins plugin: HTML Publisher Plugin ** # --8<-------------------------- separator ------------------------>8-- ** TODO [#A] jenkins get overview of scheduled jenkins jobs :IMPORTANT: ** TODO Jenkins Warnings plugin: ** TODO Jenkins Dependency-Check Plugin: ** TODO jenkins restrict user running jobs on prod env ** TODO Jenkins job: stop/start container docker pull sandbox image ** TODO Jenkins ssh key security: SSH Credentials Plugin; SSH Agent Plugin ** TODO jenkins setting: only registered user can trigger, only admin can configure ** TODO [#A] Jenkinse use a cluster for testing; jenkins slave :IMPORTANT: ** [#A] jenkins change enrinvonment variables for post-actions ** TODO [#A] jenkins docker plugin :IMPORTANT: ** TODO [#A] QA jenkins job doesn't destroy the VMs

ImagesNetworkingMonitoringAPISupport C

denny.zhang [email protected] C Carol Settings Notifications2 Logout Create kitchen-cluster-mdm-qa-4nodes-node4 4 GB Memory / 60 GB Disk / SFO2 - Ubuntu 14.04.5 x64 ipv4: Copy ipv6: Enable now Private IP: Enable now Floating IP: Enable now Console: Graphs Access Power Volumes Resize Networking Backups Snapshots Kernel History Destroy Tags ** TODO jenkins slave U can attach ur nodes as a slaves to master Jenkins or if u want to to run a command from an endpoint, u can use pac software ** TODO Jenkins Blueocean ** TODO jenkins cancel job doesn't stop db backup ** TODO [#A] jenkins cancel job doesn't force-merge command :IMPORTANT: [email protected]:/var/log/elasticsearch# ps -ef | grep el root 90 2 0 Aug11 ? 00:00:00 [khelper] root 8097 8096 0 13:50 ? 00:00:00 python /opt/devops/bin/ --espatternregexp staging-index-abae8b30ac9b11e692000401f8d88101-new3 --mindeletedcount 100000 --mindeletedratio 0 root 13315 13314 0 20:47 ? 00:00:00 python /opt/devops/bin/ --espatternregexp staging-index-abae8b30ac9b11e692000401f8d88101-new3 --mindeletedcount 100000 --mindeletedratio 0 root 20846 1844 0 22:49 pts/1 00:00:00 grep --color=auto el elastic+ 31034 1 99 Aug11 ? 36-14:05:03 /usr/lib/jvm/java-8-oracle-amd64/bin/java -Xms12288m -Xmx12288m -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -server -Djava.awt.headless=true -Xms12288m -Xmx12288m -Xss256k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-2.3.3.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d -p /var/run/elasticsearch/ --default.path.home=/usr/share/elasticsearch --default.path.logs=/var/log/elasticsearch --default.path.conf=/etc/elasticsearch ** TODO jenkins multi-stage build ** TODO jenkins multiple configuration ** TODO jenkins syntax generator that means, constantly having to look up on plugin docs for pipeline steps

4 replies Puneeth [41 minutes ago] Use the pipeline syntax generator in your Jenkins installation

amrit [40 minutes ago] Yea thats what I started using now :slightlysmilingface: . Do people still use those?

amrit [< 1 minute ago] @Denny Zhang (Github . Blogger) this can be handy for those situations

Denny Zhang (Github . Blogger) [< 1 minute ago] thanks. I haven't used it. Will give it a try ** TODO How to force jenkins to reload a jenkinsfile? ** TODO Blog: Jenkins pipeline: reconfigure jenkins job and reload it Question: How to use jenkins pipeline to reconfigure parameters of another jenkins job.

(Ideally I wish I could avoid reloading or restarting jenkins)

I remember this channel has similar discussion about this. Anyone remember the suggestion? *** TODO Jenkins pipeline: Get current setting and add a new attribute ** TODO try Jenkins X ** TODO create a jenkins job to update markdown wiki ** TODO Why jenkins are up and running, after machine reboot. But couchbase, es are not docker exec -it mdm-jenkins service jenkins status docker exec -it mdm-all-in-one bash

service couchbase-server start && service elasticsearch start

service couchbase-server status && service elasticsearch status

service mdm start

service couchbase-server status && service elasticsearch status && service mdm status ** TODO Lessons learned: run one jenkins backup ** TODO jenkins load bundle ** HALF jenkins pipeline get job configuration

dennyzhang lrpChangeMe1

curl -X GET http://dennyzhang:[email protected]:18080/job/dennytestRehearsal/config.xml -o mylocalconfig.xml

curl -X POST http://dennyzhang:[email protected]:18080/job/dennytestRehearsal/config.xml --data-binary "@mymodifiedlocalconfig.xml" ** TODO why haserror variable hasn't passed: http://myjenkins:18080/job/CheckDNSPropagation/14/console ** TODO jenkins docker image: why /var/jenkinshome/.bashrc folder is missing? ** TODO Use groovy to add a user ** TODO Use groovy to only allow register users use Jenkins ** TODO Use groovy script to restart jenkins ** TODO Use groovy to install a jenkins plugin ** # --8<-------------------------- separator ------------------------>8-- :noexport: ** TODO jenkins pipeline job: add node *** original one node { // TODO: validate users input def iplist = []; def ipportlist = []; def sshport = '2702' for (entry in iphostnamelist.split("\n")) { entry = entry.trim() ipaddress = entry.split(" ")[0] ipaddress = ipaddress.trim() iplist.add(ipaddress) ipportlist.add(ipaddress + ":" + ssh_port) }

stage('UpdateHAProxy') {
   build job: 'FixHostsFileBinding', parameters: [text(name: 'server_list', value: ''), text(name: 'add_hosts', value: ip_hostname_list)]
   build job: 'UpdateHAProxyNodeListDOBematech'
   build job: 'CheckIPAddressInList',  parameters: [text(name: 'new_ip_list', value: ip_list.join("\n"))]

stage('FixConf') { parallel firstBranch: { def target_host_file='/tmp/hosts_target' retry(2) { build job: 'GetHostFileBinding', parameters: [string(name: 'target_host_file', value: target_host_file)] } // update hosts file for existing nodes build job: 'FixHostsFileBinding', parameters: [text(name: 'add_hosts', value: ip_hostname_list)] // update hosts file for new nodes def host_binding_content=readFile(target_host_file) build job: 'FixHostsFileBinding', parameters: [text(name: 'server_list', value: ip_port_list.join("\n")), text(name: 'add_hosts', value: host_binding_content)] retry(2) { build 'FixHostsFileTemplateBematechDO' } }, secondBranch: { build job: 'UFWAddNodesBematechDO', parameters: [text(name: 'new_ip_list', value: ip_list.join("\n"))] retry(2) { build 'FixESYamlBematechDO' } retry(2) { build 'FixMDMYamlBematechDO' } }, failFast: false }

stage('Rehearsal') { if (skip_deployment_rehearsal == "false") { build job: 'DeploySystemRehearsalDOBematech', parameters: [text(name: 'server_list', value: ip_port_list.join("\n"))] } }

} ** TODO pipeline best practice ** TODO Jenkins CI Pipeline Scripts not permitted to use method groovy.lang.GroovyObject ** HALF groovy load a json file /usr/local/scripts/terraformjenkins_digitalocean/bematech-do-es-39/terraform.tfstate ** HALF groovy send http request ** TODO jenkins monitor Denny Zhang [4:47 PM] Nice, any screenshots? (Ignore, if it's against the policy)

Stefan Jansson [4:49 PM] the jenkins-plugin is called build monitor plugin:

[4:50] a competitor is radiator: - i'd use radiator once getting over a certain amount of jobs to monitor, since it has an option to only display failing jobs

[4:54] You could even call it "continuous testing in production" to make it an even stronger trend buzzword... "continuous testing" and "testing in production" are buzzwords that traditional testers do shrug from, but something I believe strongly in for the future, for devops teams, and teams running a continuous delivery process, and bigger organisations where you depend on other teams/products

Denny Zhang [4:54 PM] Yes, we have Jenkins monitor plugin enabled

[4:55] Don't quite understand its value though

Stefan Jansson [4:57 PM] what is it that you don't understand?

Denny Zhang [4:57 PM] It gives me an overview. But what I can get from it?

new messages Stefan Jansson [5:05 PM] my example works like an extra layer of monitoring, to discover even faster if you got a problem in production, if your data-logging, alarms etc might not be as fast with, or as obvious.. it won't even catch everything, that a test can.

for example, my teams builds a booking-flow for the nordics biggest travel-company.. if you cannot book a seat on a plane, simply because an API somewhere doesn't respond, or are having slow timeouts so it doesn't even display the option for the customer... the error-code monitoring and larms might go up, but it takes a while until the larms sets off, or that the error count get's to a suspicious amount so that you take a look at it.. but the TEST that runs making a booking, will fail immidiatly and give you a RED-flag on the monitor ** TODO Jenkins SCM Sync configuration plugin *** SCM Sync configuration plugin *** jenkinsfile: groovy script *** Manage Jenkins Jobs with YAML ** TODO configure pipeline status in a better way ** TODO jenkins pipeline: can't abort it ** TODO Jenkins group stage: 5 groups Romain B. [12:21 AM] @Denny Zhang (Github . Blogger): You should regroup your tests in less stages, you can still get a "test overview" in the

test result
webpage (append
to your job) Also, give BlueOcean a try, it doesn't do everything well but its pretty usefull to output pipeline status (especially to spot an error)

Dario Tranchitella [1:25 AM] Some used Dependency Injection (with Inversion of Control) and Singleton patterns using Groovy shared library? I noticed that documentation is really poor and I'm facing some scalability issues with a complex pipelines...

Denny Zhang (Github . Blogger) [8:07 AM] @romainrbr, let me give it a try ** TODO Jenkins Features Controlled with System Properties: ** TODO SCM Sync Configuration Plugin: ** TODO Explore env Jenkins update issue: updatejenkinsitself cd /var/chef/cache/

java -jar ./jenkins-cli.jar -s http://localhost:18080/ login --username chefadmin --password "TOTVS123FD" java -jar ./jenkins-cli.jar -s http://localhost:18080/ list-jobs ** TODO ip list as an inventory file provided by jenkins ** TODO Use Jenkins ssh plugin: ** TODO jenkins powershell seanturner83 [5:17 AM] @dennyzhang you like powershell? GitHub poshbotio/PoshBot PoshBot - Powershell-based bot framework ** TODO improve bematech jenkins security: about tcp ports ** TODO [#A] secure sonarqube port forwarding jenkins ** TODO [#A] jenkins pipeline fail to be aborted ** TODO jenkins pipeline specify git credential ** TODO Blog: jenkins pipeline back to normal notification ** TODO [#A] Blog: jenkins piepline update job parameter ** TODO [#A] jenkins pipeline job to update existing job ** TODO jenkins create admin user from configure ** TODO [#A] How to keep jenkins in sync for two directions? :IMPORTANT: ** TODO whenever I configure jenkins, it will crash ** TODO Candy Jenkins:


Also, the demo Jenkins you guys are running on docker?

3 replies Denny Zhang (Github . Blogger) [3 minutes ago] Jenkins is running on docker. Via AWS ECS (edited)

AnmolNagpal [1 minute ago] Oke I have a suggestion try caddy with it it's really nice and Jenkins will run on https

Denny Zhang (Github . Blogger) [< 1 minute ago] This one? GitHub david7482/caddy-jenkins-docker caddy-jenkins-docker - Host Jenkins with Caddy as https proxy in Docker


** TODO jenkins SCM Sync configuration plugin ** TODO [#A] Automatically generating Jenkins jobs *** Jenkins Job Builder Jenkins Job Builder is a command-line utility that will create Jenkins jobs based upon YAML configurations. ** TODO advanced jenkins customization Hide port: Change http://XXX.XXX.XXX.XXX:8080 to http://XXX.XXX.XXX.XXX:18080 The whole process takes more than 10 minutes, I only acccept 5 minutes Create a dedicated policy Customize EC2 profile Create Tags to manage the stack When container restart/recreate, Jenkins configuration won't be lost ** TODO Blog: how to avoid Jenkins SPOF


Denny Zhang (Github . Blogger) [10:43 AM] Let's say above 2 Jenkins instances serve the service.

As we know Jenkins have local $JENKINS_HOME folder.

So how these 2 instances work together to serve HA?

Any thoughts? @Pradipta Dash @Stefan Jansson (QA in Continuous Delivery) @Jonathan.McAllister @Keef Baker

Jonathan McAllister [10:43 AM] joined #jenkins by invitation from Denny Zhang (Github . Blogger).

Philip Schwartz [11:20 AM] @Denny Zhang (Github . Blogger) are those suppose to be 2 jenkins masters running in ecs with a load balancer infront of them?

Denny Zhang (Github . Blogger) [11:20 AM] yes

Philip Schwartz [11:20 AM] Jenkins doesn't work that way. You can't drop a load balance infront of masters as they don't cross communicate

[11:22] If they are cloudbees instances you can use the HA plugin to allow promotion on single master failure. But it is still not the same and requires shared file systems between them

Denny Zhang (Github . Blogger) [11:22 AM] How we can avoid Jenkins SPOF, Philip?

new messages Philip Schwartz [11:23 AM] With jenkins OSS there is no way


*** TODO Jenkins HA


Puneeth [12:48 PM] @Denny Zhang (Github . Blogger) jenkins master HA is essentially jenkins master in an asg with efs volume for jenkins home

[12:49] @Denny Zhang (Github . Blogger) there is a white paper from aws on jenkins HA

[12:49] jenkins slaves ha is again via asg and spot fleet

[12:49] at least that's our approach

Puneeth [12:54 PM] @Denny Zhang (Github . Blogger) there can only be one master unless unless unless you move the main job queues from the jenkins master. and use an external job queue :) In this case there can be multi master jenkins confg spread across many regions or within the same region. this is a super advanced approach. this approach is used by openstack . we used it too in our previous company



**** TODO Jenkins: **** TODO Jenkins Plugin: **** TODO [#A] Jenkins HA: ** TODO Why jenkins create user doesn't seem to work? ** TODO [#A] Blog: How chef keep as login user, even if jenkins restart ** TODO jenkins warning: Email notifications could be sent to people who are not users of Jenkins


Warnings have been published for the following currently installed components: Mailer Plugin 1.18: Email notifications could be sent to people who are not users of Jenkins


** TODO 4 Jenkins warning


You have data stored in an older format and/or unreadable data. Manage Dismiss Disable CLI over Remoting Dismiss Allowing Jenkins CLI to work in -remoting mode is considered dangerous and usually unnecessary. You are advised to disable this mode. Please refer to the CLI documentation for details. Warnings have been published for the following currently installed components: Mailer Plugin 1.18: Email notifications could be sent to people who are not users of Jenkins Go to plugin manager Configure which of these warnings are shown Examine Dismiss Agent to master security subsystem is currently off. Please read the documentation and consider turning it on


** TODO [#A] jenkins Build Pipeline plugin :IMPORTANT: ** TODO Pipeline as Code with Jenkins ** TODO try jenkinsfile-solution: backup Jenkins configuration Stefan Jansson [12:33 PM] For sure. We'll see

[12:33] You feel your jenkins-backup solution works good for you?

Denny Zhang [12:33 PM] Not very. But it's working

[12:33] ThinBackup is good. But it will introduce some problem

Stefan Jansson [12:35 PM] The jenkinsfile-solution seems pretty damn nice. I havent gone deepnintonit yet though.

You basicly points out a github-repo and its jenkinsfile. And it configures your jenkinsjob from scatch.

[12:35] Sets up, and configures, if ive understoodnitncorrectlt

Denny Zhang [12:35 PM] Yeah, I've heard of this part.

Let me give it a try. ** TODO [#A] Use Jenkinsfile instead of the UI ** TODO What's Jenkins JNLP worker? ** TODO Jenkins plugin: Build Monitor Plugin71 - Visual view of your builds status ** TODO Jenkins plugin: SSH Slaves64 - Manage workers through SSH ** TODO Jenkins plugin: Nested View Plugin70 - Groups jobs into multiple levels instead of a single big list of tabs ** TODO Jenkins plugin: Gearman

Jenkins core does not support multiple masters. ** TODO git jenkins depths issue Ken Godoy [1:38 PM] See my first screenshot

Denny Zhang (Github . Blogger) [1:38 PM] Oh yes

Ken Godoy [1:38 PM] The other options are to create a reference repo locally Or to just create a new repo as you mentioned.

Denny Zhang (Github . Blogger) [1:39 PM] hmm, the depth of 1 should be recognized. That looks weird indeed

Ken Godoy [1:39 PM] But creating a new repo I still need history for at least one year. I have automation that uses history to generate SQL script packages. I love Jenkins but sometimes it's a pain in the wazoo.

Denny Zhang (Github . Blogger) [1:40 PM] Need to deep dive into the git jenkins plugin Let me give it a try tonight as well. ** TODO When we cancel jenkins backup job, the python script won't die ** TODO try jenkins X: *** TODO [#A] why we need jenkins x *** DONE jx help CLOSED: [2018-04-23 Mon 15:46] ➜ ~ jx help

Installing: install Install Jenkins X in the current Kubernetes cluster uninstall Uninstall the Jenkins X platform upgrade Upgrades a resource create cluster Create a new kubernetes cluster create jenkins token Adds a new username and api token for a Jenkins server init Init Jenkins X

Adding Projects to Jenkins X: import Imports a local project or git repository into Jenkins create archetype Create a new app from a Maven Archetype and import the generated code into git and Jenkins for CI/CD create spring Create a new spring boot application and import the generated code into git and Jenkins for CI/CD create lile Create a new lile based application and import the generated code into git and Jenkins for CI/CD create micro Create a new micro based application and import the generated code into git and Jenkins for CI/CD create quickstart Create a new app from a Quickstart and import the generated code into git and Jenkins for CI/CD

Addons: create addon Creates an addon environment View or change the current environment in the current kubernetes cluster namespace View or change the current namespace context in the current kubernetes cluster prompt Generate the command line prompt for the current team and environment shell Create a sub shell so that changes to the kubernetes context, namespace or environment remain local to the shell status status of the Kubernetes cluster or named node

Working with Applications: console Opens the Jenkins console cdx Opens the CDX dashboard for visualising CI/CD and your environments logs Tails the log of the latest pod for a deployment open Open a service in a browser rsh Opens a terminal in a pod or runs a command in the pod

Working with Environments: preview Creates or updates a Preview Environment for the current version of an application promote Promotes a version of an application to an environment create environment Create a new Environment which is used to promote your Team's Applications via Continuous Delivery delete environment Deletes one or more environments edit environment Edits an Environment which is used to promote your Team's Applications via Continuous Delivery get environments Display one or many Environments

Working with Jenkins X resources: get Display one or many resources edit Edit a resource create Create a new resource delete Deletes one or many resources start Starts a process such as a pipeline

Jenkins X Pipeline Commands: step pipeline steps

Other Commands: help Help about any command version Print the version information Usage: jx [flags] [options] Use "jx --help" for more information about a given command. *** TODO easy how-to github token: jenkins-x

fe67390d1be344bb48b6a4d524d9ebfa167030af ** TODO Github oauth on jenkins servers ** TODO [#A] avoid wrong input parameter issues


Denny Zhang (DevOps) [10:08 PM] That's something I have done this morning.

Since we haven't update any other parts. Only the list is incorrect, so the impact is not big.

Thus I didn't mention that in the slack. But that's wrong indeed.

Robson Poffo [10:09 PM] yes, it was low impact Sandro got something wrong on some reportings and he asked me that :slightlysmilingface: all good

Denny Zhang (DevOps) [10:09 PM] Inputing wrong parameters happens sometimes. But it could be very dangerous.

I definitely need to think how to prevent this misconfigure issue! I have made a note. Will think more, and discuss with Carlos and Kung


** TODO when jenkins jobs take too long, abort it and send out alerts ** TODO How to add jenkins slave nodes in jenkins master server using Chef cookbook ** TODO Use a jenkins job to reconfigure the same chef_json for 3 jobs: DeploySystemDOBematech, DeploySystemRehearsalDOBematech, UpdateHAProxyNodeListDOBematech ** HALF copy json folder across nodes: http://myjenkins:18080/job/CopyFolderFromNode1ToNode2/ && vim

bash -ex "" "/data/staging/" "" "/tmp/staging" ** HALF [#A] doc: why vm shutdown has failed: http://myjenkins:18080/job/RunCommandOnServers/187/ ** TODO [#A] problem: Fail to shutdown couchbase nodes: http://myjenkins:18080/job/RunCommandOnServers/181/console (bematech-do-cb-008)

06:43:04 06:43:04 * couchbase-server is running 06:43:04


Denny Zhang (DevOps) [8:48 AM] So far, haven't found major issues.

All ES/mdm/haproxy services have been shutdown. All CB services have been shutdown, except one. (bematech-do-cb-008)

It has been 13 minutes since we have issued the service stop command. Will watch another 2 minutes, then "stop again". If still doesn't work, use kill, then kill -9.



[email protected]:/opt/couchbase/var/lib/couchbase/logs# service couchbase-server stop {errorlogger,{{2018,2,26},{14,48,36}},"Protocol: ~tp: the name [email protected] seems to be in use by another Erlang node",["inettcp"]} {errorlogger,{{2018,2,26},{14,48,36}},crashreport,[[{initialcall,{netkernel,init,['Argument1']}},{pid,<0.21.0>},{registeredname,[]},{errorinfo,{exit,{error,badarg},[{genserver,initit,6,[{file,"genserver.erl"},{line,320}]},{proclib,initpdoapply,3,[{file,"proclib.erl"},{line,239}]}]}},{ancestors,[netsup,kernelsup,<0.10.0>]},{messages,[]},{links,[#Port<0.53>,<0.18.0>]},{dictionary,[{longnames,true}]},{trapexit,true},{status,running},{heapsize,610},{stacksize,27},{reductions,783}],[]]} {errorlogger,{{2018,2,26},{14,48,36}},supervisorreport,[{supervisor,{local,netsup}},{errorContext,starterror},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,netkernel},{mfargs,{netkernel,startlink,[['[email protected]',longnames]]}},{restarttype,permanent},{shutdown,2000},{childtype,worker}]}]} {errorlogger,{{2018,2,26},{14,48,36}},supervisorreport,[{supervisor,{local,kernelsup}},{errorContext,starterror},{reason,{shutdown,{failedtostartchild,netkernel,{'EXIT',nodistribution}}}},{offender,[{pid,undefined},{name,netsup},{mfargs,{erldistribution,startlink,[]}},{restarttype,permanent},{shutdown,infinity},{childtype,supervisor}]}]} {errorlogger,{{2018,2,26},{14,48,36}},crashreport,[[{initialcall,{application_master,init,['Argument1','Argument2','Argument3','Argument_4']}},{pid,<0.9.0>},{registeredname,[]},{errorinfo,{exit,{{shutdown,{failedtostartchild,netsup,{shutdown,{failedtostartchild,netkernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}},[{applicationmaster,init,4,[{file,"applicationmaster.erl"},{line,133}]},{proclib,initpdoapply,3,[{file,"proclib.erl"},{line,239}]}]}},{ancestors,[<0.8.0>]},{messages,[{'EXIT',<0.10.0>,normal}]},{links,[<0.8.0>,<0.7.0>]},{dictionary,[]},{trapexit,true},{status,running},{heapsize,376},{stacksize,27},{reductions,117}],[]]} {errorlogger,{{2018,2,26},{14,48,36}},stdinfo,[{application,kernel},{exited,{{shutdown,{failedtostartchild,netsup,{shutdown,{failedtostartchild,netkernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}},{type,permanent}]} {"Kernel pid terminated",applicationcontroller,"{applicationstartfailure,kernel,{{shutdown,{failedtostartchild,netsup,{shutdown,{failedtostartchild,netkernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}}"}

Crash dump was written to: erlcrash.dump.1519656516.20531.babysitter Kernel pid terminated (applicationcontroller) ({applicationstartfailure,kernel,{{shutdown,{failedtostartchild,netsup,{shutdown,{failedtostartchild,netkernel,{'EXIT',nodistribution}}}}},{k * Failed to stop couchbase-server


** TODO jenkins: jenkins-job-builder plugin for job creation ** TODO ** TODO Jenkins: SCM Sync configuration plugin: ** TODO learn from other repo: aws jenkins:✓ ** TODO http://myjenkins:18080/job/CopyFolderFromNode1ToNode2/ ** TODO mute the output of token: http://myjenkins:18080/job/CreateAndProvisionVMInCloud/176/console ** TODO restart es: console output is very slow to show: http://myjenkins:18080/job/RestartOneESSafely/5/console ** TODO ** TODO Pull Request Validation Between Jenkins and Bitbucket: ** TODO beautify ansible output: ** TODO jenkins: ** HALF avoid jenkins: Jenkins is going to shut down Solution: disable the thinBackup plugin

Have you installed any plugin called Thin backup? if yes it might configured to shut down after back up, then change the setting. ** # --8<-------------------------- separator ------------------------>8-- :noexport: ** TODO Jenkins credential parameter ** TODO Jenkins shell get credential secret text ** TODO Jenkins parse credential parameter ** TODO Jenkins apache issue! ls -lth /var/run/apache2/ ** TODO bug: cancel backup from Jenkins won't kill the process [email protected]:/opt/couchbase/backup# ps -ef | grep cou root 30800 30799 0 18:57 ? 00:00:00 python /opt/devops/bin/ --bucketlist=mdm-session --cbserver= --cbbackupbin=/opt/couchbase/mdmpublic/couchbase-cli/bin/cbbackup --backupdir=/opt/couchbase/backup --username Administrator --password password1234 --backupmethod full root 30801 30800 0 18:57 ? 00:00:00 /bin/sh -c /opt/couchbase/mdmpublic/couchbase-cli/bin/cbbackup /opt/couchbase/backup/mdm-session -u Administrator -p password1234 -b mdm-session -m full -t 4 >> /var/log/cbbackup.log root 30802 30801 99 18:57 ? 00:00:43 python /opt/couchbase/mdmpublic/couchbase-cli/lib/python/cbbackup /opt/couchbase/backup/mdm-session -u Administrator -p password1234 -b mdm-session -m full -t 4 root 31120 12741 0 18:58 pts/4 00:00:00 grep --color=auto cou [email protected]:/opt/couchbase/backup# kill 30800 ** TODO Jenkins create a global variable: to avoid duplication configurations ** TODO Automate the thinbackup Jenkins restore ** TODO Jenkins agent and servers ** TODO automate Jenkins restore from thinbackup ** TODO High Jenkins CPU load: we can't run the jobs ** TODO Run Docker commands from Jenkins container ** TODO Use AWS S3 for my critical backup: Jenkins configurations * TODO [#A] try Jenkins X :noexport: * TODO jenkins pipeline run docker container :noexport: * TODO jenkins enable github authentication :noexport: * useful link :noexport: * # --8<-------------------------- separator ------------------------>8-- :noexport: * TODO Provision agent using Jenkins swarm plugin :noexport: * TODO Automate agent provisioning and make them ephemeral :noexport: * TODO Jenkins pipeline: docker image as agent :noexport: * TODO docker image as agent with persistent storage :noexport: * HALF Jenkins script approval :noexport:


// instance containing the approvals // list of approved hashes: println instance.approvedScriptHashes

ScriptApproval instance = Jenkins.getInstance().getExtensionList(RootAction.class).get(ScriptApproval.class); approvedScriptHashes = instance.approvedScriptHashes


  • TODO groovy: Use a regular expression to include jobs into the view :noexport:
  • # --8<-------------------------- separator ------------------------>8-- :noexport:
  • DONE [#A] Jenkins: send slack notification for job failures :noexport: CLOSED: [2019-05-01 Wed 11:06]
  • DONE [#A] jenkins kubosshpasswd: jenkins secret text :noexport: CLOSED: [2019-05-01 Wed 11:06]

  • DONE jenkins groovy get job :noexport: CLOSED: [2019-04-25 Thu 22:13] import jenkins.model.Jenkins name = "ProvisionPKSEnvRaw" def job = Hudson.instance.getJob(name) println job.getLastBuild().getLog(50)

  • DONE doc: Jenkins pipeline: if ip not found, skip healthcheck :noexport: CLOSED: [2019-04-26 Fri 19:16]

    +BEGIN_SRC groovy

    void triggerHealthCheck() { if (!kuboIP.isEmpty()) { def childjob = build job: 'HealthCheckPKSEnv', propagate: false, parameters: [string(name: 'jumpboxip', value: kuboIP), string(name: 'skipkubectltest', value: 'true')] loadAndParseJobLogs(childjob) updateBuild(childjob) } else { println("WARNING: kuboIP is not found from downstream job output. Skip health check") } }


+BEGIN_SRC groovy

    stage('Health check') {
        steps {
            echo '===> Start the health check of the provisioned nimbus testbed...'
            timeout(time: 180, unit: 'MINUTES') {
                script {


  • DONE Jenkins pipeline show user id :noexport: CLOSED: [2019-04-28 Sun 21:42] node { wrap([$class: 'BuildUser']) { def user = env.BUILDUSERID println "user: " + user } }
  • DONE Jenkins pipeline if else not working :noexport: CLOSED: [2019-04-28 Sun 22:13]

  • you can simplify this and potentially avoid the if statement (as long as you don't need the else) by using "when".

  • wrap it in a script step

+BEGIN_SRC groovy

pipeline { agent any

stages {
    stage('test') {
        steps {
            sh 'echo hello'
    stage('test1') {
        steps {
            sh 'echo $TEST'
    stage('test3') {
        steps {
            script {
                if (env.BRANCH_NAME == 'master') {
                    echo 'I only execute on the master branch'
                } else {
                    echo 'I execute elsewhere'



  • DONE jenkins credentials :noexport: CLOSED: [2019-04-29 Mon 14:45]

  • DONE jenkins fails to send slack notification: check slack plugin version: slack:2.2 :noexport: CLOSED: [2019-03-06 Wed 16:59]

  • DONE Better Jenkins UI: ocean-blue :noexport: CLOSED: [2019-03-14 Thu 16:11]

  • DONE Dynamically create jenkins users :noexport: CLOSED: [2019-03-27 Wed 11:04]

  • --8<-------------------------- separator ------------------------>8-- :noexport:

  • TODO Jenkins credential scope: Global(jenkins, nodes, items, all child items, etc) and System (Jenkins and nodes only) :noexport:

  • HALF Jenkins pipeline credentials for all stages :noexport:

  • TODO Configure jenkins slave Node :noexport:

  • TODO Semantic Versioning class for Groovy :noexport:

  • TODO [#A] Run jenkins pipeline code inside docker image :noexport:

  • TODO [#A] jenkins wrap class :noexport:

  • TODO Load groovy script as import, instead of :noexport: pipelineUtils = load "${JENKINS_HOME}/init.groovy.d/pipeline-utils.groovy"

                script {
                    pipelineUtils.sendSlackNotification(slack_target, jobLogs, downstreamJobResult)
  • HALF Jenkins pipeline check website availability issue :noexport:

  • --8<-------------------------- separator ------------------------>8-- :noexport:

  • DONE Builds in Jenkins run as the virtual SYSTEM user with full permissions by default. :noexport: CLOSED: [2019-06-13 Thu 11:18]


    Builds in Jenkins run as the virtual SYSTEM user with full permissions by default. This can be a problem if some users have restricted or no access to some jobs, but can configure others. If that is the case, it is recommended to install a plugin implementing build authentication, and to override this default. ✅ An implementation of access control for builds is present. ❌ Access control for builds is possible, but not configured. Configure it in the global security configuration


  • TODO Tried proxying jenkins.telemetry.Correlator to support a circular dependency, but it is not an interface. :noexport:

  • --8<-------------------------- separator ------------------------>8-- :noexport:

  • Jenkins cheatshee: :noexport:

  • TODO update doc :noexport:

  • Jenkins code coverage :noexport: ** Cobertura coverage report * jenkins groovy timeout set the value :noexport: * TODO Integrate GitLab MR with Jenkins Job in your CI Pipeline :noexport:'ttriggerbuildsonAcceptedMergeRequests


Create a Pipeline Job and configure Build Trigger as follows

Please Note that "Filter merge request by label" is optional. If you add any label, this job will be triggered only when the MR contains these labels Generate Secret toke by clicking "Generate" button. (Please make note of the token, we need to enter same value in the GitLab hook in Step 3) Configure your Pipeline job location. (Jenkins Job will scan the git-repo that you configure for "Jenkinsfile" file)

Credentials : You can use whoever have atleast "developer" permission on this project. Branches to build: "${gitlabSourceRepoName}/${gitlabSourceBranch}" Script path: This is the path to the Jenkinsfile in your repo. Save your pipeline job. Now, its time to configure the webhook in GitLab repo to trigger this job on MR. Go to your repo in GitLab → Settings → Integrations

Enter the token that you generated in your Jenkins pipeline job and enable all appropriate checkboxes for your project. Add GitLab logger in Jenkins Go to Jenkins -> Manage Jenkins -> System Log Add new log recorder Enter 'Gitlab plugin' or whatever you want for the name On the next page, enter 'com.dabsquared.gitlabjenkins' for Logger, set log level to FINEST, and save Then click on your Gitlab plugin log, click 'Clear this log' if necessary, and then use GitLab to trigger some actions Refresh the log page and you should see output Go back to the Integration page in the GitLab, click on "Test" button to see if your webhook is triggering the Jenkins job. you can just see the log in the Jenkins under the logger you just created in previous step Send a Dummy MR to the branch (Make sure you add the "labels" if you enabled any filtering in your job) and see if the pipeline is getting triggered, if not, check the logs in the Jenkins and Gitlab (by editing the hook, you can see the Recent Deliveries) More info: More info on writing pipeline stages with GitLab:


  • TODO jenkins pipeline use array :noexport:
  • TODO jenkins groovy format string :noexport:
  • groovy :noexport: String fileContents = "zdenny-vdnet-esx-sb-30035046-1-poc-dev-denny-22 | zdenny-vdnet-esx-sb-30035046-3-poc-dev-denny-22 | zdenny-vdnet-esx-sb-30035046-2-poc-dev-denny-22 | zdenny-vdnet-esx-sb-30035046-100-poc-dev-denny-22 | zdenny-vdnet-vc-sb-30035088-1-poc-dev-denny-22 | zdenny-vdnet-nsxmanager-14880419-1-poc-dev-denny-22 | zdenny-wn-1-poc-dev-denny-22";

def digitPattern = ~/.(vdnet-vc)./;

def matcher = fileContents =~ digitPattern; matcher.size(); matcher[0] matcher[1]

// digitPattern.matcher(fileContents).find()


We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.