Migrating old documentation from Microsoft Excel into Atlassian Confluence I was bamboozled that I couldn’t simply copy paste excel data into Confluence Tables. Duh!

Googling around for a couple of minutes a provided solution looked like that:

• Convert Excel to markdown
• Use the “Insert markdown” macro in Confluence

Excel to markdown

There are several ways to convert Excel to markdown:

Online Converters

There are quite a few only converters e.g.
https://www.tablesgenerator.com/markdown_tables
but beware: when dealing with sensitive company data you might want to avoid these!

IDE integration

For Visual Studio Code there is a neat extension:
https://marketplace.visualstudio.com/items?itemName=csholmq.excel-to-markdown-table

Some Python

Another way is to use pandas to take care of the conversion:

import pandas as pd
df = pd.read_excel('test.xlsx')
md_table = df.to_markdown(index=False)
with open("test.md", "w") as f:
    f.write(md_table)

When we are dealing with multiple sheets inside our excel document we can do the following:

sheet_names = pd.ExcelFile('test.xlsx').sheet_names
for sheet in sheet_names:
    df = pd.read_excel('test.xlsx', sheet)
    md_table = df.to_markdown(index=False)
    print(md_table)
    with open(f"test_{sheet}.md", "w") as f:
        f.write(md_table)

The post Convert Excel Sheets to Confluence Tables appeared first on Creatronix.

Getting the AWS Cloud Practitioner Certificate is one of my goals from my New Year’s Resolutions and Technology Learning Roadmap 2023
So time to buckle up and learn about cloud concepts first!

The Cloud

tl;dr; the cloud is someone else’s computer 🙂

In general cloud computing means that you don’t pay for computer hardware like servers, NAS, SAN devices anymore.
Instead, you use them as a service.

Service Models

There are a couple of different service models.

IaaS

Renting other people’s servers and storage units is called Infrastructure as a service or IaaS.
Instead of installing components into a well-protected, air-conditioned server room on your company’s premise you pay the cloud provider for doing so.
Advantages: You trade capital expense with operational expenses.
Disadvantages: operating costs of data centers are also comprised of maintaining the software components.
So another service model is called Platform as a service or PaaS

PaaS

In this use case you pay for hardware usage and updates of the system as well. This is called Platform as a Service (PaaS)
The PaaS provider will take care of updating the OS and other dedicated packages (think: sudo apt-get)
You will only have to keep your applications up to date.

SaaS

Last but not least: SaaS or Software as Service
In many cases you want to use a specific software and don’t want to be concerned with keeping the application itself upto-date
(thinks databases) So there is a service model where you just use a specific software and don’t have to worry about updates / maintenance at all.

FaaS

Ah, hold on, one more thing. There is a so called “serverless” concept.
Serverless vendors offer compute runtimes, also known as Function as a Service (FaaS)
The cloud provider allocates machine resources on demand, taking care of the servers on behalf of their customers.
An example is AWS Lambda

Further Reading

https://aws.amazon.com/types-of-cloud-computing/

The post Cloud Computing Overview appeared first on Creatronix.

Inspired by automate the boring stuff and motivated by my last automation attempts, I wanted to see if there is more stuff which can be automated.
Creating YouTube thumbnails is one of the time consuming tasks I tried to simplify.

Installation

Pillow is a fork of PIL the Python Imaging Library. We can install it with:

pip install pillow

Loading images

def test_open_image():
    thumbnail = Image.open("example_img.jpg")
    assert thumbnail.size == (4032, 3024)

Resizing Images

YouTube recommends 1200×720 as the standard thumbnail size. So let’s use pillow to resize the image:

thumbnail = Image.open("example_img.jpg").convert("RGB")
thumbnail.thumbnail((1200, 720), Image.LANCZOS)
thumbnail.save("example_img_resized.jpg")

Loading Fonts

The TrueType font format was developed by Apple and Microsoft as a response to the PostScript font format.
TTF has long been the most common format for fonts on Mac and Windows operating systems.

title_font = ImageFont.truetype(r'c:\windows\fonts\arial.ttf')

If you are sure that the font is a registered system font you don’t need to provide the absolute path

title_font = ImageFont.truetype('arial.ttf')

On macOs you will encounter another related format which is called ttc

title_font = ImageFont.truetype("Supplemental/Futura.ttc")

Font size

The default size of the loaded font is 10. With the size parameter you can adjust that:

title_font = ImageFont.truetype("Supplemental/Futura.ttc", size=120)

Drawing text on images

thumbnail = Image.open("example_img.jpg")
if sys.platform == "win32":
    title_font = ImageFont.truetype("arial.ttf", size=120)
else:
    title_font = ImageFont.truetype("Supplemental/Futura.ttc", size=120)
draw_surface = ImageDraw.Draw(thumbnail, 'RGBA')
draw_surface.text((0, 0), "Hello Pillow", font=title_font, fill=(148, 14, 76, 0))
thumbnail.save("thumbnail_with_text.jpg")

For adding text we need to add a drawing surface to the image with the Draw function. The text function will draw the text to the surface. The fill parameter takes an RGBA tuple for the font color.

Centering text

To center the text in x-axis we need the width of the image and the width of the textbox

thumbnail = Image.open("example_img.jpg")
thumbnail_width = thumbnail.size[0]

The image object has a size attribute. The width is at index 0

left, top, right, bottom = title_font.getbbox("Hello Pillow")

With the getbbox function we can get the coordinates of the left top and the right bottom of the text’s bounding box so we have to calculate the width with:

textbox_width = right - left

Finally we can calculate the starting position of the textbox:

textbox_start_x = (thumbnail_width - textbox_width) / 2

Complete Code

from PIL import Image, ImageFont, ImageDraw
def test_add_centered_text_to_image():
    thumbnail = Image.open("example_img.jpg")
    draw_surface = ImageDraw.Draw(thumbnail)
    title_font = ImageFont.truetype("arial.ttf", size=120)
    thumbnail_width = thumbnail.size[0]
    left, top, right, bottom = title_font.getbbox("Hello Pillow")
    textbox_width = right - left
    textbox_start_x = (thumbnail_width - textbox_width) / 2
    textbox_start_y = 80
    draw_surface.text((textbox_start_x, textbox_start_y), "Hello Pillow", font=title_font, fill=PG_GREEN)
    thumbnail.save("thumbnail_with_centered_text.jpg")

The post How to create youtube thumbnails – pillow tutorial appeared first on Creatronix.

If you had less menial manual work you could sit at the beach and relax more often 🙂

So regarding my pain points I figured out that publishing articles to my blog takes a huge amount of time too.

In order to publish an article I need to:

• Log in into the WordPress backend
• Create a new post in the WordPress backend
• Insert the markdown text for the article into the post (I write my articles in markdown and use github for versioning)
• Convert the markdown format into HTML by pressing the enter key on every tag (sic!) (so that ## becomes h2)
• Upload all media / images
• Insert images into the post
• Select the featured image
• Write some SEO relevant metadata
• Publish the post

Why not “Automate the boring stuff” here as well? In this part I will show you the first part of publishing a post.

Prerequisites

For uploading we use the requests module. So let’s install it via:

pip install requests

Application Password

As another prerequisites we need an application password:

• Log in to your WordPress site with an admin user account
• Navigate to Users > Profile.
• Scroll down to the “Application Passwords” heading.
• Enter an application name
• Click “Add New Application Password”
• Store the generated password in a secure place

Uploading Posts

data = {
    'title': 'Example wordpress post',
    'status': 'draft',
    'slug': 'example-wordpress-post',
    'author': 1,
    'content': '# H1 heading \n ## h2 heading'
}

This is the minimal data you have to provide to be able to create a post via API.
The author has to be an integer. You can find the users’ ids via https://<your_wordpress_site>/wp-json/wp/v2/users in your browser.

Authentication

For authentication you need a header:

WORDPRESS_CREDENTIALS = USER_NAME + ":" + APPLICATION_PASSWORD 
WORDPRESS_TOKEN = base64.b64encode(WORDPRESS_CREDENTIALS.encode()) 
WORDPRESS_HEADER = {'Authorization': 'Basic ' + WORDPRESS_TOKEN.decode('utf-8')}

USER_NAME has to be a real user with admin rights not the application password name (It got me confused)

Posting the Post

We use the REST API v2 which is located at:

WORDPRESS_API = "https://<your_wordpress_site>/wp-json/wp/v2"

The endpoint for uploading posts is

API_ENDPOINT = WORDPRESS_API + "/posts"

Now we have all information to finally post the post

def upload_post(data):
    response = requests.post(API_ENDPOINT, json=data, headers=WORDPRESS_HEADER)
    return response

Little helpers

Creating slugs

Normally the slug for a post is the title just with lowercase and hyphen instead of spaces
(That’s what the editor in the wp backend would normally do automatically)
So let’s do this programmatically as well:

def make_slug_from_title(title):
    slug = title.lower().replace(" ", "-")
    return slug

Converting from markdown to HTML

With the markdown package you can convert your article before uploading

pip install Markdown

html_text = markdown(article_content, output_format="html")

Just one line of additional code. I love Python!

Reading and Converting a markdown file

To create the data for the upload in one go I wrote a convenience function:

def get_article_data(folder):
    base_dir = '../content'
    article = os.path.join(base_dir, folder, folder + ".md")
    with open(article, "r") as f:
        title = f.readline().replace("#", "").strip()
        article_content = f.read()
    html_text = markdown(article_content, output_format="html")
    slug = make_slug_from_title(title)
    data = {
        'title': title,
        'status': 'draft',
        'slug': slug,
        'author': 1,
        'content': html_text,
    }
    return data

Putting it together

With these helpers you can condense it into

data = get_article_data(folder)
upload_post(data)

Further Reading

How to use Automator on macOS

How to use google translate with python

https://developer.wordpress.org/rest-api/

The post How to create WordPress articles via REST API appeared first on Creatronix.

Inspired by the session at BarCamp Regensburg I started to think about pain points in my workflow. I figured out that the conversion of my photos was too cumbersome.

Before

When writing articles I download the images from my phone via AirDrop to my Mac. Apple stores images in the HEIC format on my iPhone. I cannot upload HEIC directly to my blog. So I need to convert them with Preview by opening and exporting them to jpg. This is not very elegant especially when you have a lot of images.

Automation

So let’s see if we can use Automator to make our lives easier

Open Automator

Go to Launchpad and Open Automator

This is the main screen:

Quick Action

Select “Quick Action” from menu

Select Action

Search for “Change Types of Images”

Drag & Drop it to the right panel and set Output Image Type by setting the “To Type” dropdown to JPEG

Export

Open the Menu File -> Export

Name the workflow e.g. HEIC_to_JPG_conversion.workflow

Save it to the desktop

Installation

Open Finder and go to the desktop and double click. Click “Install”

Another window will open and show you the active extension including your workflow:

Usage

When you open up a context menu (right-click in Finder) you will now see your workflow

Deleting Workflows

Go to ~/Library/Services
Delete the workflow you do not need any more

Further Reading

Pillow how to convert images to webp

How to create youtube thumbnails &#8211; pillow tutorial

The post How to use Automator on macOS appeared first on Creatronix.

Sometimes you need to translate text from one language to another.
I for example often write articles in German but want to have them in English for this blog.
For this use case you can use google’s translation function in the browser.
But wouldn’t it be sweet to automate this task and spare some copy/paste orgies?

Installation

pip install googletrans==3.1.0a0

First Translation – German to English

def test_translate_de_en():
    from googletrans import Translator
    translator = Translator()
    text = "Wie geht's?"
    translated = translator.translate(text, src="de", dest="en")
    assert translated.text == "How are you?"

English to German

def test_translate_en_de():
    from googletrans import Translator
    translator = Translator()
    text = 'How are you?'
    translated = translator.translate(text, src='en', dest="de")
    assert translated.text == 'Wie geht es dir?'

Limitations

The maximum character limit on a single text is 15,000

The post How to use google translate with python appeared first on Creatronix.

When writing Python packages you often have to assure that the packages can be used on different platforms and Python versions

e.g. the package must work under Python 3.8 on Linux as well as with Python 3.9 on Windows.

But how do you test all those supported OS and version combinations?

Enter tox

History

tox was first released in October 2018, first stable version 0.5 came out in 2010.

Today we have the 3.25 and tox 4 is in the making.

Installation

pip install tox

Setup & Usage

At a bare minimum you need a tox.ini file in your project’s root directory with the follwing content:

[tox]
envlist = py38, py39

[testenv]
deps = pytest
commands = pytest

To run tox you just type in

tox

and Bob’s your uncle

How does it work?

tox will now do the following things automagically for you

1. Build the package (Make an sdist distribution by invoking python setup.py sdist)
2. For each listed environment (in this example Python 3.8 and 3.9)
   1. Create the environment with venv
   2. Install the package
   3. Install additional dependencies
   4. Run the commands

tox will create a .tox directory in your project

More configuration

Running on multiple OS

[tox]
envlist = {win32, linux}-{py38, py39}

[testenv]
platform = win32
deps = -r ./requirements-dev.txt
commands = coverage run -m pytest --junit-xml=reports/test_report-{envname}.xml

[testenv:linux-py38]
platform = linux|linux2

[testenv:linux-py39]
platform = linux|linux2

tox has a neat syntax to create a matrix of OS and Python versions

{win32, linux}-{py38, py39} will result in the following environments

win32-py38
win32-py39
linux-py38
linux-py39

Building wheels

If you are interested in testing the creation of wheels as well you can use a plugin like

https://pypi.org/project/tox-wheel/

or call python setup.py bdist_wheel directly in the commands section

Python 3.10

[tox]
envlist = py37, py38, py39, py310

Multiple dependencies

If your tox builds rely on more dependencies you have to list them  like this:

[testenv]
deps = pytest
       pyhamcrest

But to avoid duplication you should use the dependencies from your requirements file

[testenv]
deps = -r ./requirements-dev.txt

Testing Coverage

When you add coverage and pytest-cov to your requirements you can generate test coverage

[testenv]
deps = -r ./requirements-dev.txt
commands = coverage run -m pytest --junit-xml=reports/test_report-{envname}.xml

Pitfalls

WARNING: could not copy distfile to D:\.tox\distshare

drive not mounted

Further Reading

https://tox.wiki/en/latest/index.html

https://github.com/jboegeholz/easypattern/tree/wip

The post How to use tox to test your code on multiple platforms appeared first on Creatronix.

Sometimes you encounter websites which seem to have time traveled from 1999. No responsive UI, no widgets, just plain html input fields and selects.

Time to improve these sites 🙂

Greasemonkey overview

Greasemonkey is a browser plugin which lets you inject client-side JavaScript into the webpage just as the developer could have intended to do.

GM for firefox

If you use Google Chrome you can use the Tampermonkey extension.

Now you can manipulate the DOM to make the page look more appealing and function better.

Hello World Greasemonkey

Let’s start with a simple script:

// ==UserScript==
// @name hello_greasemonkey
// @namespace https://creatronix.de
// @description prints "Hello Greasemonkey" to the console
// @include https://creatronix.de*
// @version 1
// ==/UserScript== 
console.log("Hello Greasemonkey");

This script does nothing more than printing a string to the console

Ctrl + Shift + J opens the Browser console in Firefox.

Hint: If you don’t see the console output you have to activate “Show Content messages” in the gear menu of the browser console.

The @include statement ties the script to a specific URL. The wildcard * can be used to indicate all sub-URLs shall be included as well.

But wait – there is more:

Add jQuery UI Datepicker

I’ve had a page where the date input where just plain input fields.

With the following script I’ve added a nice date picker widget.

For that purpose you can use jQuery and jQuery UI.

With the @require you can use a CDN to import the needed libraries.

// ==UserScript==
// @name add_calendar_widget
// @namespace creatronix
// @description adds an jQuery-UI calendar to date fields
// @include <the_site_to_insert_the_script>
// @require https://code.jquery.com/jquery-3.6.0.min.js
// @require https://code.jquery.com/ui/1.13.1/jquery-ui.min.js
// @version 1

// ==/UserScript==

$(document).ready(function(){
  console.log("script works");
  $( 'input[name="from"]' ).datepicker({ firstDay: 1, dateFormat: 'dd.mm.yy' });
  $( 'input[name="until"]' ).datepicker({ firstDay: 1, dateFormat: 'dd.mm.yy' });
  $("#ui-datepicker-div").css({'background-color':'white'});
}

The post How to use Greasemonkey to make your life easier appeared first on Creatronix.

The biggest advantage of a Jenkinsfile is the ability to use version control to manage your build system.

When you use freestyle jobs you can loose you configuration due to limit history depth.

Descriptive vs Scripted

Jenkinsfiles come in two flavours:

Declarative pipelines always begin with the word pipeline

Scripted pipelines always begin with the word node

This tutorial deals with the declarative pipeline scripts.

Prerequisites

Before you can use a Jenkinsfile-Job you need to install this plugin in your Jenkins:

https://plugins.jenkins.io/workflow-aggregator/

A minimalistic Jenkinsfile

pipeline {
    agent any

    environment{

    }
    stages {
        stage('Build') {
            steps {
                echo 'Building..'
            }
        }
        stage('Test') {
            steps {
                echo 'Testing..'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying....'
            }
        }
    }
}

Agent

The agent defines on which build node the project can be build.

agent any

‘any’ means that the job can be run on every available node on your Jenkins master. You can of course tie a job to a certain node e.g. by label

agent { label 'windows'}

Environment

The environment block can be used to set environment variables e.g. path to git executable, special directories for reports.

environment{
    NEW_VERSION = '1.3.0'
}

Stages & Steps

You can divide your pipeline script into several logical units called stages.

e.g. Build – Test – Deploy

You can add additional stages for linting, quality checking, code coverage and so on.

The steps is where the magic happens. A stage can consist of several steps

Conditions

stage('Deploy') {

    when {
        expression {
            BRANCH_NAME == 'main'
        }
    }

 steps { echo 'Deploying....' } }

You can apply a when block to tie some stages to a specific preconditions. In this example we only want to deploy our project when we are dealing with the main branch.

Parallel execution

If you want to execute tests in parallel you do the following:

stage('Test') {
    parallel {
        stage('Test On Windows') {
            agent {
                label "windows"
            }
            steps {
                bat "run-tests.bat"
            }
        }
        stage('Test On Linux') {
            agent {
                label "linux"
            }
            steps {
                sh "run-tests.sh"
            }
        }
    }
}

Post actions

post {
    always {
    }
    failure {
    }
}

Available Environment Variables

https://<your_jenkins_ip_orcname>/env-vars.html

e.g. https://localhost:8080/env-vars.html

IDE / Tool Support

If you want IDEA to recognize a Jenkinsfile as a Groovy file, then you can add the String “Jenkinsfile” as a valid file name pattern (normally contains file endings) for Groovy files. This is supported “out of the box” without requiring any additional Plugin (except the “Groovy” Plugin, but that is already part of IDEA).

Source: https://stackoverflow.com/questions/47796757/jenkinsfile-syntax-highlighting-in-java-project-using-intellij-idea/48326267#48326267

Another solution for PyCharm:

The following filetype xml will highlight Jenkinsfile syntax in PyCharm

https://stackoverflow.com/questions/51124878/jenkinsfile-syntax-highlighting-in-pycharm-idea/56749404#56749404

Another way is to use the Pipeline Editor Plugin

The post Anatomy of a Jenkinsfile – Tutorial appeared first on Creatronix.

As long as your project is open source aka not private you can use Travis CI for free (with a limited quota of 10000 minutes)

Account

Create an account at travis-ci.com 

Connect it with your github account

Configuration

For enable travis CI in your project you must add a .travis.yml file to the root folder. Mind the dot for hidden file!

The content for a python project can look as follows:

language: python
python:
  - "3.7"
  - "3.8"
  - "3.9"
  - "3.10"

script: 
  - python -m unittest discover

Under the python tag you can list all versions under which your code shall be executed. If you don’t provide further information the testing platform is Ubuntu 16.04 Xenial. If you want to run things on Ubuntu 20.04 you can add:

dist: focal

Optional Badge

You can add a badge to your readme

[![Build Status](https://app.travis-ci.com/jboegeholz/easypattern.svg?branch=master)](https://app.travis-ci.com/jboegeholz/easypattern)

Further Reading

https://docs.travis-ci.com/user/languages/python/

The post How to add Travis CI to your github project appeared first on Creatronix.