Tag python

I have been wanting to use my Heroku account for a while with something a little more interesting than a Jupiter Notebook.

I was hoping to try and do something with Django … but there’s a lot to using Django. I have some interesting things I’m doing on my local machine, but it’s not quite ready yet.

I had googled to find other Python Web frameworks and saw that Bottle was an even more light weight framework than Flask, so I thought, hey, maybe I can do something with that.

I found this tutorial on how to do something relatively simple with Bottle and deploying to Heroku. Just what I wanted!

I got through to the end of the tutorial and deployed to Heroku. The terminal output from the Heroku command indicated that a newer version of Python (3.7.3) was available than the one I was on (3.7.1).

I figured it would be easy enough to upgrade to the newest version of Python on my Mac because I had done it before.

I don’t know why I thought the virtual environment would be different than the local install of Python 3 but it turns out they are more tightly coupled than I thought.

Upgrading to 3.7.3 broke the virtual environment I had in PyCharm. I did a bit a googling to see how to upgrade a virtual environment and found nothing. Like literally nothing.

It was ... disheartening. But after a good night’s sleep I had a thought! What if I just delete the virtual environment directory and then recreated it.

I ran this command to remove the virtual environment:

rm -R venv

Then created a virtual environment in PyCharm and now I have 3.7.3 in my virtual environment.

I had to make some changes to the files for deployment to Heroku, but that’s all covered in the tutorial mentioned above.

Sometimes the answer is to just restart it … and sometimes the answer is delete it and start over.

Update

I was listening to an episode of Python Bytes and heard Michael Kennedy (of Talk Python to Me fame) describing basically the same issue I had. Turns out, he solved it the same way I did. Nice to know i’m In good company.

Posted by ryan on Thu 25 April 2019

Sometimes the internet is a horrible, awful, ugly thing. And then other times, it’s exactly what you need.

I have 2 Raspberry Pi each with different versions of Python. One running python 3.4.2 and the other running Python 3.5.3. I have previously tried to upgrade the version of the Pi running 3.5.3 to a more recent version (in this case 3.6.1) and read 10s of articles on how to do it. It did not go well. Parts seemed to have worked, while others didn’t. I have 3.6.1 installed, but in order to run it I have to issue the command python3.6 which is fine but not really what I was looking for.

For whatever reason, although I do nearly all of my Python development on my Mac, it hadn’t occurred to me to upgrade Python there until last night.

With a simple Google search the first result came to Stackoverflow (what else?) and this answer.

brew update
brew upgrade python3

Sometimes things on a Mac do ‘just work’. This was one of those times.

I’m now running Python 3.7.1 and I’ll I needed to do was a simple command in the terminal.

God bless the internet.

Posted by ryan on Sat 22 December 2018

I was listening to the most recent episode of ATP and John Siracusa mentioned a programmer test called fizz buzz that I hadn’t heard of before.

I decided that I’d give it a shot when I got home using Python and Bash, just to see if I could (I was sure I could, but you know, wanted to make sure).

Sure enough, with a bit of googling to remember some syntax of Python, and learn some syntax for bash, I had two stupid little programs for fizz buzz.

Python

def main():

    my_number = input("Enter a number: ")

    if not my_number.isdigit():
        return
    else:
        my_number = int(my_number)
        if my_number%3 == 0 and my_number%15!=0:
            print("fizz")
        elif my_number%5 == 0 and my_number%15!=0:
            print("buzz")
        elif my_number%15 == 0:
            print("fizz buzz")
        else:
            print(my_number)


if __name__ == '__main__':
    main()

Bash

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

#! /bin/bash

echo "Enter a Number: "

read my_number

re='^[+-]?[0-9]+$'
if ! [[ $my_number =~ $re ]] ; then
   echo "error: Not a number" >&2; exit 1
fi

if ! ((my_number % 3)) && ((my_number % 15)); then
    echo "fizz"
elif ! ((my_number % 5)) && ((my_number % 15)); then
    echo "buzz"
elif ! ((my_number % 15)) ; then
    echo "fizz buzz"
else
    echo my_number
fi

And because if it isn’t in GitHub it didn’t happen, I committed it to my fizz-buzz repo.

I figure it might be kind of neat to write it in as many languages as I can, you know … for when I’m bored.

Posted by ryan on Wed 28 November 2018

It’s time for Kings Hockey! A couple of years ago Emily and I I decided to be Hockey fans. This hasn’t really meant anything except that we picked a team (the Kings) and ‘rooted’ for them (i.e. talked sh*t* to our hockey friends), looked up their position in the standings, and basically said, “Umm ... yeah, we’re hockey fans.”

When the 2018 baseball season ended, and with the lack of interest in the NFL (or the NBA) Emily and I decided to actually focus on the NHL. Step 1 in becoming a Kings fan is watching the games. To that end we got a subscription to NHL Center Ice and have committed to watching the games.

Step 2 is getting notified of when the games are on. To accomplish this I added the games to our family calendar, and decided to use what I learned writing my ITFDB program and write one for the Kings.

For the Dodgers I had to create a CSV file and read it’s contents. Fortunately, the NHL as a sweet API that I could use. This also gave me an opportunity to use an API for the first time!

The API is relatively straight forward and has some really good documentation so using it wasn’t too challenging.

import requests
from sense_hat import SenseHat
from datetime import datetime
import pytz
from dateutil.relativedelta import relativedelta



def main(team_id):
    sense = SenseHat()

    local_tz = pytz.timezone('America/Los_Angeles')
    utc_now = pytz.utc.localize(datetime.utcnow())
    now = utc_now.astimezone(local_tz)

    url = 'https://statsapi.web.nhl.com/api/v1/schedule?teamId={}'.format(team_id)
    r = requests.get(url)

    total_games = r.json().get('totalGames')

    for i in range(total_games):
        game_time = (r.json().get('dates')[i].get('games')[0].get('gameDate'))
        away_team = (r.json().get('dates')[i].get('games')[0].get('teams').get('away').get('team').get('name'))
        home_team = (r.json().get('dates')[i].get('games')[0].get('teams').get('home').get('team').get('name'))
        away_team_id = (r.json().get('dates')[i].get('games')[0].get('teams').get('away').get('team').get('id'))
        home_team_id = (r.json().get('dates')[i].get('games')[0].get('teams').get('home').get('team').get('id'))
        game_time = datetime.strptime(game_time, '%Y-%m-%dT%H:%M:%SZ').replace(tzinfo=pytz.utc).astimezone(local_tz)
        minute_diff = relativedelta(now, game_time).minutes
        hour_diff = relativedelta(now, game_time).hours
        day_diff = relativedelta(now, game_time).days
        month_diff = relativedelta(now, game_time).months
        game_time_hour = str(game_time.hour)
        game_time_minute = '0'+str(game_time.minute)
        game_time = game_time_hour+":"+game_time_minute[-2:]
        away_record = return_record(away_team_id)
        home_record = return_record(home_team_id)
        if month_diff == 0 and day_diff == 0 and hour_diff == 0 and 0 >= minute_diff >= -10:
            if home_team_id == team_id:
                msg = 'The {} ({}) will be playing the {} ({}) at {}'.format(home_team, home_record, away_team, away_record ,game_time)
            else:
                msg = 'The {} ({}) will be playing at the {} ({}) at {}'.format(home_team, home_record, away_team, away_record ,game_time)
            sense.show_message(msg, scroll_speed=0.05)


def return_record(team_id):
    standings_url = 'https://statsapi.web.nhl.com/api/v1/teams/{}/stats'.format(team_id)
    r = requests.get(standings_url)
    wins = (r.json().get('stats')[0].get('splits')[0].get('stat').get('wins'))
    losses = (r.json().get('stats')[0].get('splits')[0].get('stat').get('losses'))
    otl = (r.json().get('stats')[0].get('splits')[0].get('stat').get('ot'))
    record = str(wins)+'-'+str(losses)+'-'+str(otl)
    return record


if __name__ == '__main__':
    main(26) # This is the code for the LA Kings; the ID can be found here: https://statsapi.web.nhl.com/api/v1/teams/

The part that was the most interesting for me was getting the opponent name and then the record for both the opponent and the Kings. Since this is live data it allows the records to be updated which I couldn’t do (easily) with the Dodgers programs (hey MLB ... anytime you want to have a free API I’m ready!).

Anyway, it was super fun and on November 6 I had the opportunity to actually see it work:

<iframe src="https://www.youtube.com/embed/AzdLSrA8wvU" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"></iframe>

I really like doing fun little projects like this.

Posted by ryan on Fri 09 November 2018

Every once in a while I get a wild hair and decide that I need to ‘clean up’ my directories. This never ends well and I almost always mess up something, but I still do it.

Why? I’m not sure, except that I forget that I’ll screw it up. 🤦‍♂️

Anyway, on a Saturday morning when I had nothing but time I decided that I’d move my PyCharm directory from /Users/ryan/PyCharm to /Users/ryan/Documents/PyCharm for no other reason than because.

I proceeded to use the command line to move the folder

mv /Users/ryan/PyCharm/ /Users/ryan/Documents/PyCharm/

Nothing too big, right. Just a simple file movement.

Not so much. I then tried to open a project in PyCharm and it promptly freaked out. Since I use virtual environments for my Python Project AND they tend to have paths that reference where they exist, suddenly ALL of my virtual environments were kind of just gone.

Whoops!

OK. No big deal. I just undid my move

mv /Users/ryan/Documents/PyCharm/ /Users/ryan/PyCharm

That should fix me up, right?

Well, mostly. I had to re-register the virtual environments and reinstall all of the packages in my projects (mostly not a big deal with PyCharm) but holy crap it was scary. I thought I had hosed my entire set of projects (not that I have anything that’s critical … but still).

Anyway, this is mostly a note to myself.

The next time you get a wild hair to move stuff around, just keep it where it is. There’s no reason for it (unless there is).

But seriously, ask yourself first, “If I don’t move this what will happen?” If the answer is anything less than “Something awful” go watch a baseball game, or go to the pool, or write some code. Don’t mess with your environment unless you really want to spend a couple of hours unmasking it up!

Posted by ryan on Sun 12 August 2018

A Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text.

Uses include:

1. data cleaning and transformation
2. numerical simulation
3. statistical modeling
4. data visualization
5. machine learning
6. and other stuff

I’ve been interested in how to set up a Jupyter Notebook on my Linode server for a while, but kept running into a roadblock (either mental or technical I’m not really sure).

Then I came across this ‘sweet’ solution to get them set up athttp://blog.lerner.co.il/five-minute-guide-setting-jupyter-notebook-server/

My main issue was what I needed to to do keep the Jupyter Notebook running once I disconnected from command line. The solution above gave me what I needed to solve that problem

nohup jupyter notebook

nohup allows you to disconnect from the terminal but keeps the command running in the background (which is exactly what I wanted).

The next thing I wanted to do was to have the jupyter notebook server run from a directory that wasn’t my home directory.

To do this was way easier than I thought. You just run nohup jupyter notebook from the directory you want to run it from.

The last thing to do was to make sure that the notebook would start up with a server reboot. For that I wrote a shell script

# change to correct directory
cd /home/ryan/jupyter

nohup jupyter notebook &> /home/ryan/output.log

The last command is a slight modification of the line from above. I really wanted the output to get directed to a file that wasn’t in the directory that the Jupyter notebook would be running from. Not any reason (that I know of anyway) … I just didn’t like the nohup.out file in the working directory.

Anyway, I now have a running Jupyter Notebook at http://python.ryancheley.com:8888¹

1. I’d like to update this to be running from a port other than 8888 AND I’d like to have it on SSL, but one thing at a time! ↩︎

Posted by ryan on Sun 27 May 2018

After days of trying to figure this out, I finally got the video to upload via a cronjob.

There were 2 issues.

Issue the first

Finally found the issue. Original script from YouTube developers guidehad this:

CLIENT_SECRETS_FILE = "client_secrets.json"

And then a couple of lines later, this:

% os.path.abspath(os.path.join(os.path.dirname(__file__), CLIENT_SECRETS_FILE))

When crontab would run the script it would run from a path that wasn’t where the CLIENT_SECRETS_FILE file was and so a message would be displayed:

WARNING: Please configure OAuth 2.0

To make this sample run you will need to populate the client_secrets.json file
found at:

  %s

with information from the Developers Console
https://console.developers.google.com/

For more information about the client_secrets.json file format, please visit:
https://developers.google.com/api-client-library/python/guide/aaa_client_secrets

What I needed to do was to update the CLIENT_SECRETS_FILE to be the whole path so that it could always find the file.

A simple change:

CLIENT_SECRETS_FILE  = os.path.abspath(os.path.join(os.path.dirname(__file__), CLIENT_SECRETS_FILE))

Issue the second

When the create_mp4.sh script would run it was reading all of the h264 files from the directory where they lived BUT they were attempting to output the mp4 file to / which it didn’t have permission to write to.

This was failing silently (I’m still not sure how I could have caught the error). Since there was no mp4 file to upload that script was failing (though it was true that the location of the CLIENT_SECRETS_FILE was an issue).

What I needed to do was change the create_mp4.sh file so that when the MP4Box command output the mp4 file to the proper directory. The script went from this:

(echo '#!/bin/sh'; echo -n "MP4Box"; array=($(ls ~/Documents/python_projects/*.h264)); for index in ${!array[@]}; do if [ "$index" -eq 0 ]; then echo -n " -add ${array[index]}"; else echo -n " -cat ${array[index]}"; fi; done; echo -n " hummingbird.mp4") > create_mp4.sh

To this:

(echo '#!/bin/sh'; echo -n "MP4Box"; array=($(ls ~/Documents/python_projects/*.h264)); for index in ${!array[@]}; do if [ "$index" -eq 0 ]; then echo -n " -add ${array[index]}"; else echo -n " -cat ${array[index]}"; fi; done; echo -n " /home/pi/Documents/python_projects/hummingbird.mp4") > /home/pi/Documents/python_projects/create_mp4.sh

The last bit /home/pi/Documents/python_projects/create_mp4.sh may not be necessary but I’m not taking any chances.

The video posted tonight is the first one that was completely automatic!

Now … if I could just figure out how to automatically fill up my hummingbird feeder.

Posted by ryan on Fri 20 April 2018

I’ve mentioned before that I have been working on getting the hummingbird video upload automated.

Each time I thought I had it, and each time I was wrong.

For some reason I could run it from the command line without issue, but when the cronjob would try and run it ... nothing.

Turns out, it was running, it just wasn’t doing anything. And that was my fault.

The file I had setup in cronjob was called run_scrip.sh

At first I was confused because the script was suppose to be writing out to a log file all of it’s activities. But it didn’t appear to.

Then I noticed that the log.txt file it was writing was in the main \` directory. That should have been my first clue.

I kept trying to get the script to run, but suddenly, in a blaze of glory, realized that it was running, it just wasn’t doing anything.

And it wasn’t doing anything for the same reason that the log file was being written to the \` directory.

All of the paths were relative instead of absolute, so when the script ran the command ./create_mp4.sh it looks for that script in the home directory, didn’t find it, and moved on.

The fix was simple enough, just add absolute paths and we’re golden.

That means my run_script.sh goes from this:

# Create the script that will be run
./create_script.sh
echo "Create Shell Script: $(date)" >> log.txt

# make the script that was just created executable
chmod +x /home/pi/Documents/python_projects/create_mp4.sh

# Create the script to create the mp4 file
/home/pi/Documents/python_projects/create_mp4.sh
echo "Create MP4 Shell Script: $(date)" >> /home/pi/Documents/python_projects/log.txt

# upload video to YouTube.com
/home/pi/Documents/python_projects/upload.sh
echo "Uploaded Video to YouTube.com: $(date)" >> /home/pi/Documents/python_projects/log.txt

# Next we remove the video files locally
rm /home/pi/Documents/python_projects/*.h264
echo "removed h264 files: $(date)" >> /home/pi/Documents/python_projects/log.txt

rm /home/pi/Documents/python_projects/*.mp4
echo "removed mp4 file: $(date)" >> /home/pi/Documents/python_projects/log.txt

To this:

# change to the directory with all of the files
cd /home/pi/Documents/python_projects/

# Create the script that will be run
/home/pi/Documents/python_projects/create_script.sh
echo "Create Shell Script: $(date)" >> /home/pi/Documents/python_projects/log.txt

# make the script that was just created executable
chmod +x /home/pi/Documents/python_projects/create_mp4.sh

# Create the script to create the mp4 file
/home/pi/Documents/python_projects/create_mp4.sh
echo "Create MP4 Shell Script: $(date)" >> /home/pi/Documents/python_projects/log.txt

# upload video to YouTube.com
/home/pi/Documents/python_projects/upload.sh
echo "Uploaded Video to YouTube.com: $(date)" >> /home/pi/Documents/python_projects/log.txt

# Next we remove the video files locally
rm /home/pi/Documents/python_projects/*.h264
echo "removed h264 files: $(date)" >> /home/pi/Documents/python_projects/log.txt

rm /home/pi/Documents/python_projects/*.mp4
echo "removed mp4 file: $(date)" >> /home/pi/Documents/python_projects/log.txt

I made this change and then started getting an error about not being able to access a json file necessary for the upload to YouTube. Sigh.

Then while searching for what directory the cronjob was running from I found this very simple idea. The response was, why not just change it to the directory you want. 🤦‍♂️

I added the cd to the top of the file:

# change to the directory with all of the files
cd /home/pi/Documents/python_projects/

Anyway, now it works. Finally!

Tomorrow will be the first time (unless of course something else goes wrong) that The entire process will be automated. Super pumped!

Posted by ryan on Tue 10 April 2018

I previously wrote about how I placed my Raspberry Pi above my hummingbird feeder and added a camera to it to capture video.

Well, the day has finally come where I’ve been able to put my video of it up on YouTube! It’s totally silly, but it was satisfying getting it out there for everyone to watch and see.

Hummingbird Video Capture: Addendum

The code used to generate the the mp4 file haven’t changed (really). I did do a couple of things to make it a little easier though.

I have 2 scripts that generate the file and then copy it from the pi to my MacBook Pro and the clean up:

Script 1 is called create_script.sh and looks like this:

(echo '#!/bin/sh'; echo -n "MP4Box"; array=($(ls *.h264)); for index in ${!array[@]}; do if [ "$index" -eq 0 ]; then echo -n " -add ${array[index]}"; else echo -n " -cat ${array[index]}"; fi; done; echo -n " hummingbird.mp4") > create_mp4.sh | chmod +x create_mp4.sh

This creates a script called create_mp4.sh and makes it executable.

This script is called by another script called run_script.sh and looks like this:

./create_script.sh
./create_mp4.sh

scp hummingbird.mp4 ryan@192.168.1.209:/Users/ryan/Desktop/

# Next we remove the video files locally

rm *.h264
rm *.mp4

It runs the create_script.sh which creates create_mpr.sh and then runs it.

Then I use the scp command to copy the mp4 file that was just created over to my Mac Book Pro.

As a last bit of housekeeping I clean up the video files.

I’ve added this run_script.sh to a cron job that is scheduled to run every night at midnight.

We’ll see how well it runs tomorrow night!

Posted by ryan on Thu 05 April 2018

Last Wednesday if you would have asked what I had planned for Easter I would have said something like, “Going to hide some eggs for my daughter even though she knows the Easter bunny isn’t real.”

Then suddenly my wife and I were planning on entertaining for 11 family members. My how things change!

Since I was going to have family over, some of whom are Giants fans, I wanted to show them the ITFDB program I have set up with my Pi.

The only problem is that they would be over at 10am and leave by 2pm while the game doesn’t start until 5:37pm (Thanks ESPN).

To help demonstrate the script I wrote a demo script to display a message on the Pi and play the Vin Scully mp3.

The Code was simple enough:

from sense_hat import SenseHat
import os


def main():
    sense = SenseHat()
    message = '#ITFDB!!! The Dodgers will be playing San Francisco at 5:37pm tonight!'
    sense.show_message(message, scroll_speed=0.05)
    os.system("omxplayer -b /home/pi/Documents/python_projects/itfdb/dodger_baseball.mp3")


if __name__ == '__main__':
    main()

But then the question becomes, how can I easily launch the script without futzing with my laptop?

I knew that I could run a shell script for the Workflow app on my iPhone with a single action, so I wrote a simple shell script

python3 ~/Documents/python_projects/itfdb/demo.py

Which was called itfdb_demo.sh

And made it executable

chmod u+x itfdb_demo.sh

Finally, I created a WorkFlow which has only one action Run Script over SSH and added it to my home screen so that with a simple tap I could demo the results.

The WorkFlow looks like this:

Nothing too fancy, but I was able to reliably and easily demonstrate what I had done. And it was pretty freaking cool!

Posted by ryan on Sun 01 April 2018