RyanCheley.com - Tag bash

Styling Clean Up with Bash

I have a side project I've been working on for a while now. One thing that happened overtime is that the styling of the site grew organically. I'm not a designer, and I didn't have a master set of templates or design principals guiding the development. I kind of hacked it together and made it look "nice enough"

That was until I really starting going from one page to another and realized that there styling of various pages wasn't just a little off ... but A LOT off.

As an aside, I'm using tailwind as my CSS Framework

I wanted to make some changes to the styling and realized I had two choices:

Manually go through each html template (the project is a Django project) and catalog the styles used for each element

Try and write a bash command to do it for me

Well, before we jump into either choice, let's see how many templates there are to review!

As I said above, this is a Django project. I keep all of my templates in a single templates directory with each app having it's own sub directory.

I was able to use this one line to count the number of html files in the templates directory (and all of the sub directories as well)

ls -R templates | grep html | wc -l

There are 3 parts to this:

ls -R templates will list out all of the files recursively list subdirectories encountered in the templates directory
grep html will make sure to only return those files with html
wc -l uses the word, line, character, and byte count to return the number of lines return from the previous command

In each case one command is piped to the next.

This resulted in 41 html files.

OK, I'm not going to want to manually review 41 files. Looks like we'll be going with option 2, "Try and write a bash command to do it for me"

In the end the bash script is actually relatively straight forward. We're just using grep two times. But it's the options on grep that change (as well as the regex used) that are what make the magic happen

The first thing I want to do is find all of the lines that have the string class= in them. Since there are html templates, that's a pretty sure fire way to find all of the places where the styles I am interested in are being applied

I use a package called djhtml to lint my templates, but just in case something got missed, I want to ignore case when doing my regex, i.e, class= should be found, but so should cLass= or Class=. In order to get that I need to have the i flag enabled.

Since the html files may be in the base directory templates or one of the subdirectories, I need to recursively search, so I include the r flag as well

This gets us

grep -ri "class=" templates/*

That command will output a whole lines like this:

templates/tasks/steps_lists.html:    <table class="table-fixed w-full border text-center">
templates/tasks/steps_lists.html:                <th class="w-1/2 flex justify-left-2 p-2">Task</th>
templates/tasks/steps_lists.html:                <th class="w-1/4 justify-center p-2">Edit</th>
templates/tasks/steps_lists.html:                <th class="w-1/4 justify-center p-2">Delete</th>
templates/tasks/steps_lists.html:                    <td class="flex justify-left-2 p-2">
templates/tasks/steps_lists.html:                    <td class="p-2 text-center">
templates/tasks/steps_lists.html:                        <a class="block hover:text-gray-600"
templates/tasks/steps_lists.html:                            <i class="fas fa-edit"></i>
templates/tasks/steps_lists.html:                    <td class="p-2 text-center">
templates/tasks/steps_lists.html:                        <a class="block hover:text-gray-600"
templates/tasks/steps_lists.html:                            <i class="fas fa-trash-alt"></i>
templates/tasks/step_form.html:        <section class="bg-gray-400 text-center py-2">
templates/tasks/step_form.html:            <button type="submit" class="bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded">{{view.action|default:"Add"}} </button>

Great! We have the data we need, now we just want to clean it up.

Again, we'll use grep only this time we want to look for an honest to goodness regular expression. We're trying to identify everything in between the first open angle brackey (<) and the first closed angle bracket (>)

A bit of googling, searching stack overflow, and playing with the great site regex101.com gets you this

<[^\/].*?>

OK, we have the regular expression we need, but what options do we need to use in grep? In this case we actually have two options:

Use egrep (which allows for extended regular expressions)
Use grep -E to make grep behave like egrep

I chose to go with option 2, use grep -E. Next, we want to return ONLY the part of the line that matches the regex. For that, we can use the option o. Putting it all together we get

grep -Eo "<[^\/].*?>"

Now, we can pipe the results from our first command into our second command and we get this:

grep -ri "class=" templates/* | grep -Eo "<[^\/].*?>"

This will output to standard out, but next I really want to use a tool for aggregation and comparison. It was at this point that I decided the best next tool to use would be Excel. So I sent the output to a text file and then opened that text file in Excel to do the final review. To output the above to a text file called tailwind.txt we

grep -ri "class=" templates/* | grep -Eo "<[^\/].*?>" > tailwind.txt

With these results I was able to find several styling inconsistencies and then fix them up. In all it took me a few nights of working out the bash commands and then a few more nights to get the styling consistent. In the process I learned so much about grep and egrep. It was a good exercise to have gone through.

Posted by ryan on Tue 26 October 2021

Monitoring the temperature of my Raspberry Pi Camera

In late April of this year I wrote a script that would capture the temperature of the Raspberry Pi that sits above my Hummingbird feeder and log it to a file.

It’s a straight forward enough script that captures the date, time and temperature as given by the internal measure_temp function. In code it looks like this:

MyDate="`date +'%m/%d/%Y, %H:%M, '`"
MyTemp="`/opt/vc/bin/vcgencmd measure_temp |tr -d "=temp'C"`"
echo "$MyDate$MyTemp" >> /home/pi/Documents/python_projects/temperature/temp.log

I haven’t ever really done anything with the file, but one thing I wanted to do was to get alerted if (when) the temperature exceeded the recommended level of 70 C.

To do this I installed ssmtp onto my Pi using apt-get

sudo apt-get install ssmtp

With that installed I am able to send an email using the following command:

echo "This is the email body" | mail -s "This is the subject" user@domain.tld

With this tool in place I was able to attempt to send an alert if (when) the Pi’s temperature got above 70 C (the maximum recommended running temp).

At first, I tried adding this code:

if [ "$MyTemp" -gt "70" ]; then
   echo "Camera Pi Running Hot" | mail -s "Warning! The Camera Pi is Running Hot!!!" user@domain.tld
fi

Where the $MyTemp came from the above code that gets logged to the temp.log file.

It didn’t work. The problem is that the temperature I’m capturing for logging purposes is a float, while the item it was being compared to was an integer. No problem, I’ll just make the “70” into a “70.0” and that will fix the ... oh wait. That didn’t work either.

OK. I tried various combinations, trying to see what would work and finally determined that there is a way to get the temperature as an integer, but it meant using a different method to capture it. This is done by adding this line:

ComparisonTemp=$(($(cat /sys/class/thermal/thermal_zone0/temp)/1000))

The code above gets the temperature as an integer. I then use that in my if statement for checking the temperature:

if [ "$ComparisonTemp" -gt "70" ]; then
   echo "Camera Pi Running Hot" | mail -s "Warning! The Camera Pi is Running Hot!!!" user@domain.tld
fi

Giving a final script that looks like this:

MyDate="`date +'%m/%d/%Y, %H:%M, '`"
MyTemp="`/opt/vc/bin/vcgencmd measure_temp |tr -d "=temp'C"`"
echo "$MyDate$MyTemp" >> /home/pi/Documents/python_projects/temperature/temp.log
ComparisonTemp=$(($(cat /sys/class/thermal/thermal_zone0/temp)/1000))

if [ "$ComparisonTemp" -gt "70" ]; then
   echo "Camera Pi Running Hot" | mail -s "Warning! The Camera Pi is Running Hot!!!" user@domain.tld
fi

Posted by ryan on Tue 04 December 2018

Fizz Buzz

I was listening to the most recent episode of ATP and John Siracusa mentioned a programmer test called fizz buzz that I hadn’t heard of before.

I decided that I’d give it a shot when I got home using Python and Bash, just to see if I could (I was sure I could, but you know, wanted to make sure).

Sure enough, with a bit of googling to remember some syntax of Python, and learn some syntax for bash, I had two stupid little programs for fizz buzz.

Python

def main():

    my_number = input("Enter a number: ")

    if not my_number.isdigit():
        return
    else:
        my_number = int(my_number)
        if my_number%3 == 0 and my_number%15!=0:
            print("fizz")
        elif my_number%5 == 0 and my_number%15!=0:
            print("buzz")
        elif my_number%15 == 0:
            print("fizz buzz")
        else:
            print(my_number)


if __name__ == '__main__':
    main()

Bash

#! /bin/bash

echo "Enter a Number: "

read my_number

re='^[+-]?[0-9]+$'
if ! [[ $my_number =~ $re ]] ; then
   echo "error: Not a number" >&2; exit 1
fi

if ! ((my_number % 3)) && ((my_number % 15)); then
    echo "fizz"
elif ! ((my_number % 5)) && ((my_number % 15)); then
    echo "buzz"
elif ! ((my_number % 15)) ; then
    echo "fizz buzz"
else
    echo my_number
fi

And because if it isn’t in GitHub it didn’t happen, I committed it to my fizz-buzz repo.

I figure it might be kind of neat to write it in as many languages as I can, you know … for when I’m bored.

Posted by ryan on Wed 28 November 2018

Using MP4Box to concatenate many .h264 files into one MP4 file: revisited

In my last post I wrote out the steps that I was going to use to turn a ton of .h264 files into one mp4 file with the use of MP4Box.

Before outlining my steps I said, “The method below works but I’m sure that there is a better way to do it.”

Shortly after posting that I decided to find that better way. Turns out, ~~it wasn’t really that much more work~~ it was much harder than originally thought.

The command below is a single line and it will create a text file (com.txt) and then execute it as a bash script:

~~(echo '#!/bin/sh'; for i in *.h264; do if [ "$i" -eq 1 ]; then echo -n " -add $i"; else echo -n " -cat $i"; fi; done; echo -n " hummingbird.mp4") > /Desktop/com.txt | chmod +x /Desktop/com.txt | ~/Desktop/com.txt~~

(echo '#!/bin/sh'; echo -n "MP4Box"; array=($(ls *.h264)); for index in ${!array[@]}; do if [ "$index" -eq 1 ]; then echo -n " -add ${array[index]}"; else echo -n " -cat ${array[index]}"; fi; done; echo -n " hummingbird.mp4") > com.txt | chmod +x com.txt

Next you execute the script with

./com.txt

OK, but what is it doing? The parentheses surround a set of echo commands that output to com.txt. I’m using a for loop with an if statement. The reason I can’t do a straight for loop is because the first h264 file used in MP4Box needs to have the -add flag while all of the others need the -cat flag.

Once the file is output to the com.txt file (on the Desktop) I pipe it to the chmod +x command to change it’s mode to make it executable.

Finally, I pipe that to a command to run the file ~/Desktop/com.txt

I was pretty stoked when I figured it out and was able to get it to run.

The next step will be to use it for the hundreds of h264 files that will be output from my hummingbird camera that I just installed today.

I’ll have a post on that in the next couple of days.

Posted by ryan on Sat 10 February 2018

Using MP4Box to concatenate many .h264 files into one MP4 file

The general form of the concatenate command for MP4Box is:

MP4Box -add <filename>.ext -cat <filename>.ext output.ext¹

When you have more than a couple of output files, you’re going to want to automate that -cat part as much as possible because let’s face it, writing out that statement even more than a couple of times will get really old really fast.

The method below works but I’m sure that there is a better way to do it.

echo out the command you want to run. In this case:

(echo -n "MP4Box "; for i in *.h264; do echo -n " -cat $i"; done; echo -n " hummingbird.mp4") >> com.txt

Edit the file com.txt created in (1) so that you can change the first -cat to -add

vim com.txt

While still in vim editing the com.txt file add the #!/bin/sh to the first line. When finished, exit vim²
Change the mode of the file so it can run

chmod +x com.txt

Run the file:

./com.txt

Why am I doing all of this? I have a Raspberry Pi with a Camera attachment and a motion sensor. I’d like to watch the hummingbirds that come to my hummingbird feeder with it for a day or two and get some sweet video. We’ll see how it goes.

The -add will add the \<filename> to the output file while the -cat will add any other files to the output file (all while not overwriting the output file so that the files all get streamed together). ↩︎
I’m sure there’s an xkcd comic about this, but I just can’t find it! ↩︎

Posted by ryan on Thu 08 February 2018