My first project after completing the 100 Days of Web in Python

As I mentioned in my last post, after completing the 100 Days of Web in Python I was moving forward with a Django app I wrote.

I pushed up my first version to Heroku on August 24. At that point it would allow users to add a game that they had seen, but when it disaplyed the games it would show a number (the game’s ID) instead of anything useful.

A few nights ago (Aug 28) I committed a version which allows the user to see which game they add, i.e. there are actual human readable details versus just a number!

The page can be found here. It feels really good to have it up in a place where people can actually see it. That being said I discovered a a couple of things on the publish that I’d like to fix.

I have a method that returns details about the game. One problem is that if any of the elements return None then the front page returns a Server 500 error … this is not good.

It took a bit of googling to see what the issue was. The way I found the answer was to see an idea to turn Debug to True on my ‘prod’ server and see the output. That helped me identify the issue.

To ‘fix’ it in the short term I just deleted all of the data for the games seen in the database.

I’m glad that it happened because it taught me some stuff that I knew I needed to do, but maybe didn’t pay enough attention to … like writing unit tests.

Based on that experience I wrote out a roadmap of sorts for the updates I want to get into the app:

  • Tests for all classes and methods
  • Ability to add minor league games
  • Create a Stadium Listing View
  • More robust search tool that allows a single team to be selected
  • Logged in user view for only their games
  • Create a List View of games logged per stadium
  • Create a List View of attendees (i.e. users) at games logged
  • Add more user features:
    • Ability to add a picture
    • Ability to add Twitter handle
    • Ability to add Instagram handle
    • Ability to add game notes
  • Create a Heroku Pipeline to ensure that pushes to PROD are done through a UAT site
  • Create a blog (as a pelican standalone sub domain)

It’s a lot of things but I’ve already done some things that I wanted to:

  • Added SSL
  • Set up to go to actual domain instead of Heroku subdomain

I’ll write up how I did the set up for the site so I can do it again. It’s not well documented when your registrar is Hover and you’ve got your site on Heroku. Man … it was an tough.

My Experience with the 100 Days of Web in Python

As soon as I discovered the Talk Python to me Podcast, I discovered the Talk Python to me courses. Through my job I have a basically free subscription to PluralSight so I wasn’t sure that I needed to pay for the courses when I was effectively getting courses in Python for free.

After taking a couple ( well, truth be told, all ) of the Python courses at PluralSight, I decided, what the heck, the courses at Talk Python looked interesting, Michael Kennedy has a good instructor’s voice and is genuinely excited about Python, and if it didn’t work out, it didn’t work out.

I’m so glad I did, and I’m so glad I went through the 100 Days of Web in Python course.

On May 2, 2019 I saw that the course had been released and I tweeted

This x 1000000! Thank you so much @TalkPython. I can’t wait to get started!

I started on the course on May 4, 2019 and completed it August 11, 2019. Full details on the course are here.

Of the 28 concepts that were reviewed over the course, my favorites things were learning Django and Django Rest Framework and Pelican. Holy crap, those parts were just so much fun for me. Part of my interest in Django and DRF comes from William S Vincent’s books and Podcast Django Chat, but having actual videos to watch to get me through some of the things that have been conceptually tougher for me was a godsend.

The other part that I really liked was actual deployment to a server. I had tried (about 16 months ago) to deploy a Django app to Digital Ocean and it was an unmitigated disaster. No static files no matter what I did. I eventually gave up.

In this course I really learned how to deploy to both Heroku and a Linux box on Digital Ocean, and so now I feel much more confident that the app I’m working on (more on that below) will actually see the light of day on something other than a dev machine!

The one thing that I started to build (and am continuing to work on) is an app with a DRF backend and a Vue.js front end that allows a user to track which Baseball stadia they’ve been to. So far I have an API set up via DRF (hosted at Heroku) and sketches of what to do in Vue.js. There’s also a Django front end (but it’s not the solution I really want to use).

Writing code for 100 days is hard. Like really hard. For nearly 20 of those days I was on a family vacation in the Mid Western part of the US, but I made time for both the coding, and my family. My family was super supportive of my goal which was helpful, but the content in the course was really interesting and challenging and made me want to do it every day, which was also super helpful.

On day 85 I got a video from Bob that helped get me through the last 2 weeks. It was encouraging, and helpful which is just what I needed. So thank you Bob.

At the end I also got a nice congratulatory video from Julian, which was surprising to say the least, especially because he called out some of the things that I tweeted that I enjoyed about the class, addressed me by name, and just genuinely made me feel good about my accomplishment!

OK. I just wrapped up the 100 Days of Code with Python and the web. Now what?

I took a week off to recuperate and am now ready to ‘get back to it’.

After all, I’ve got baseball stadia to track in my app!

Talk Python to me Podcast

Why I like the Talk Python Podcast

When I started listening to it

Listening to the back catalog (nearly all of it)

Epic Family Road trip – 2019 edition

My daughter Abby was in the Robotics class at school this year. This gave her (and us as a family) the opportunity to go to the Global Conference on Educational and Robotics which was held in Norman, Oklahoma.

Being in Oklahoma we had a golden opportunity to road trip from Oklahoma back to home in California, so we did.

The trip went like this:

Fly from San Diego to Oklahoma City via Phoenix. Once we landed we were in the Oklahoma City / Norman area for a week as Abby competed in GCER.

While there, Emily and I were able explore quite a bit visiting Down Town Norman very nearly every day we were there. The neatest part of the Oklahoma segment was our drive down to Washington, OK where Emily’s grand father was born (or spent time as a child … I’m not really sure).

Once we left Oklahoma we started the road trip in earnest. I’ve tried to create a Google Maps version of the trip, but the number of places we stopped is more than you can enter into a trip in Google maps.

Here are the vital statistics:

  • miles driven: 3730
  • cities visited: 17
  • national parks visited: 7
  • Baseball games seen: 3

And here are the details:

  • Norman, OK
  • Joplin, MO
  • St. Louis, MO
  • Hermann, MO
  • Jefferson City, MO
  • Kansas City, MO
  • Omaha, NE
  • Sioux Falls, SD
  • De Smet, SD
  • Pierre, SD
  • Black Hills, SD
  • Box Elder, SD
  • Rapid City, SD
  • Jewel Cave
  • Wind Cave
  • Hot Springs, SD
  • Cheyenne, WY
  • Greely, CO
  • Denver, CO
  • Grand Junction, CO
  • Arches National Park, UT
  • Cedar City, UT

We got to watch the OKC Dodgers, St. Louis Cardinals, and Kansas City Royals all play and in each case the home team won. This was good because none of the MLB teams we saw were playing the LA Dodgers, and it’s always fun to see the home team win.

Finally, I also learned some things on the trip:

  • There’s a ton of stuff to do in Norman
  • Missouri is really into World War I and its kind of weird
  • Omaha is the Silicon Valley of the midwest … so much so that they call it the Silicon Prairie
  • Denver isn’t actually in the mountains. It’s just really high in the Great Plains on the way to the Rockies
  • Grand Junction In NOT a mountain town
  • Cedar City is more than just the little Main Street that I’ve seen before … we stayed at a farm while we were there

The family is all glad to be home, and tomorrow it’s back to normal life. I have to say, I’m really looking forward to it.

Taking Down the RPi Camera Over My Hummingbird Feeder

As the temperature heats up it’s time to take down my hummingbird feeder. While the winds have cooled down the valley for the last few days, 100+ days are slowly creeping in and I need to take it down before the CPU melts.

When I took it down last year I though, meh, how hard could it be to put back up. And then I put it back up in the Fall last year and had nothing but problems.

This year, I wanted to document the wires and what not so that I can just put it back up once the temps cool down outside.

Anyway, I could describe it or just take some pictures … so here are some pictures for when I need to set it up again later this year.

Above the feeder:

Wires to the sensor:

Wires to the GPIO pins:

Figuring out how Drafts REALLY works

On my way back from Arizona a few weeks ago I decided to play around with Drafts a bit. Now I use Drafts every day. When it went to a subscription model more than a year ago it was a no brainer for me. This is a seriously powerful app when you need it.

But since my initial workflows and shortcuts I’ve not really done too much with it. But after listening to some stuff from Tim Nahumck I decided I needed to invest a little time … and honestly there’s no better time than cruising at 25k feet on your way back from Phoenix.

Ok, first of all I never really understood workspaces. I had some set up but I didn’t get it. That was the first place I started.

Each workspace can have its own action and keyboard shortcut thing which I didn’t realize. This has so much potential. I can create workspaces for all sorts of things and have the keyboard shortcut things I need when I need them! This alone is mind blowing and I’m disappointed I didn’t look into this feature sooner.

I have 4 workspaces set up:

  • OF Templates
  • O3
  • Scrum
  • post ideas

Initially since I didn’t really understand the power of the workspace I had them mostly as filtering tools to be used when trying to find a draft. But now with the custom action and keyboards for each workspace I have them set up to filter down to specific tags AND use their own keyboards.

The OF Template workspace is used to create OmniFocus projects based on Taskpaper markup. There are a ton of different actions that I took from Rose Orchard (of Automators fame) that help to either add items with the correct syntax to a Task Paper markdown file OR turn the whole thing into an OmniFocus project. Simply a life saver for when I really know all of the steps that are going to be involved in a project and I want to write them all down!

The O3 workspace is used for processing the notes from the one-on-one I have with my team. There’s really only two actions: Parse O3 notes and Add to O3 notes. How are these different? I have a Siri Shortcut that populates a Draft with a template that collects the name of the person and the date time that the O3 occurred. This is the note that is parsed by the first action. The second action is used when someone does something that I want to remember (either good or bad) so that I can bring it up at a more appropriate time (the best time to tell someone about a behavior is right now, but sometimes circumstances prevent that) so I have this little action.

In both cases they append data to a markdown file in Dropbox (i have one file per person that reports to me). The Shortcut also takes any actions that need to be completed and adds them to OmniFocus for me to review later.

The third workspace is Scrum. This workspace has just one action which is “Parse scrum notes”. Again, I have a template that is generated from Siri Shortcuts and dropped into Drafts. During the morning standup meetings I have with my team this Draft will have the things I did yesterday, what I’m working on today, and any roadblocks that I have. It also create a section where I can add actions which when the draft is parsed goes into OmniFocus for me to review later (currently the items get added with a due date of today at 1pm … but I need to revisit that).

The last workspace is post ideas (which is where I’m writing this from). Its custom keyboard is just a markdown one with quick ways to add markdown syntax and a Preview button so I can see what the markdown will render out as.

It’s still a work in progress as this draft will end up in Ulysses so it can get posted to my site, but I’ve seen that I can even post from Drafts to WordPress so I’m going to give that a shot later on.

There are several other ideas I have bouncing around in my head about ideas for potential workspaces. My only concern at this point is how many workspaces can I have before there are too many to be used effectively.

So glad I had the time on the flight to take a look at workspaces. A huge productivity boost for me!

Upgrading Python in a Virtual Environment

I have been wanting to use my Heroku account for a while with something a little more interesting than a Jupiter Notebook.

I was hoping to try and do something with Django … but there’s a lot to using Django. I have some interesting things I’m doing on my local machine, but it’s not quite ready yet.

I had googled to find other Python Web frameworks and saw that Bottle was an even more light weight framework than Flask, so I thought, hey, maybe I can do something with that.

I found this tutorial on how to do something relatively simple with Bottle and deploying to Heroku. Just what I wanted!

I got through to the end of the tutorial and deployed to Heroku. The terminal output from the Heroku command indicated that a newer version of Python (3.7.3) was available than the one I was on (3.7.1).

I figured it would be easy enough to upgrade to the newest version of Python on my Mac because I had done it before.

I don’t know why I thought the virtual environment would be different than the local install of Python 3 but it turns out they are more tightly coupled than I thought.

Upgrading to 3.7.3 broke the virtual environment I had in PyCharm. I did a bit a googling to see how to upgrade a virtual environment and found nothing. Like literally nothing.

It was … disheartening. But after a good night’s sleep I had a thought! What if I just delete the virtual environment directory and then recreated it.

I ran this command to remove the virtual environment:

rm -R venv

Then created a virtual environment in PyCharm and now I have 3.7.3 in my virtual environment.

I had to make some changes to the files for deployment to Heroku, but that’s all covered in the tutorial mentioned above.

Sometimes the answer is to just restart it … and sometimes the answer is delete it and start over.

Update

I was listening to an episode of Python Bytes and heard Michael Kennedy (of Talk Python to Me fame) describing basically the same issue I had. Turns out, he solved it the same way I did. Nice to know i’m In good company.

Did you try restarting it?

The number of times an issue is resolved with a simple reboot is amazing. It’s why when you call tech support (for anything) it’s always the first thing they ask you.

Even with my experience in tech I can forget this one little trick when troubleshooting my own stuff. I don’t have a tech support line to call so I have to google, and google and google, and since the assumption is that I’ve already rebooted, it’s not a standard answer that’s put out there. (I mean, of course I rebooted to see if that fixed the problem).

I’ve written before about my ITFDB and the announcement from Vin Scully “It’s Time for Dodger Baseball!”. With the start of the 2019 season the mp3 stopped playing.

I tried all sorts of fixes. I made sure the Pi was up to date with apt-get update and apt-get upgrade. I thought maybe the issue was due to the version of Python running on the Pi (3.4.2). I thought maybe the mp3 had become corrupt and tried to regenerate it.

None of these things worked. Finally I found this post and the answer was so obvious. To quote the answer:

Have you tried rebooting?

It’s a total shot in the dark, but I just transitioned from XBMC to omxplayer and lost sound. What I did:

apt-get remove xbmc

apt-get autoremove

apt-get update

apt-get upgrade

After that I lost sound. 10 minutes of frustration later I rebooted and everything worked again.

It wasn’t exactly my problem, but upon seeing it I decided “What the hell?” And you know what, it totally worked.

I wish I would have checked to see when the last time a reboot had occurred, but it didn’t occur to me until I started writing this post. Oh well … it doesn’t really matter because it works now.

Creating Hastags for Social Media with a Drafts Action

Creating meaningful, long #hastags can be a pain in the butt.

There you are, writing up a witty tweet or making that perfect caption for your instagram pic and you realize that you have a fantastic idea for a hash tag that is more of a sentence than a single word.

You proceed to write it out and unleash your masterpiece to the world and just as you hit the submit button you notice that you have a typo, or the wrong spelling of a word and #ohcrap you need to delete and retweet!

That lead me to write a Drafts Action to take care of that.

I’ll leave others to write about the virtues of Drafts, but it’s fantastic.

The Action I created has two steps: (1) to run some JavaScript and (2) to copy the contents of the draft to the Clipboard. You can get my action here.

Here’s the JavaScript that I used to take a big long sentence and turn it into a social media worthy hashtag

var contents = draft.content;
var newContents = "#";


editor.setText(newContents+contents.replace(/ /g, "").toLowerCase());

Super simple, but holy crap does it help!

Making it easy to ssh into a remote server: Addendum

I recently got a new raspberry pi (yes, I might have a problem) and wanted to be able to ssh into it without having to remember the IP or password. Luckily I wrote this helpful post several months ago.

While it go me most of the way there, I did run into a slight issue.

First Issue

The issue was that I had a typo for the command to generate a key. I had:

ssh-keyken -t rsa

Which should have been:

ssh-keygen -t rsa

When I copied and pasted the original command the terminal said there was no such command. 🤦‍♂️

Second Issue

Once that go cleared up I went through the steps and was able to get everything set up. Or so I thought. On attempting to ssh into my new pi I was greeted with a password prompt. WTF?

The first thing I did was to check to see what keys were in my ~/.ssh folder. Sure enough there were a couple of them in there.

ls ~/.ssh
id_rsa             id_rsa.github      id_rsa.github.pub  id_rsa.pub         known_hosts        read_only_key      read_only_key.pub

Next, I interrogated the help command for ssh-copy-id to see what flags were available.

Usage: /usr/bin/ssh-copy-id [-h|-?|-f|-n] [-i [identity_file]] [-p port] [[-o <ssh -o options>] ...] [user@]hostname
	-f: force mode -- copy keys without trying to check if they are already installed
	-n: dry run    -- no keys are actually copied
	-h|-?: print this help

I figured let’s try the -n flag and get the output from that. Doing so gave me

ryan@Ryans-MBP:~/Desktop$ ssh-copy-id -n pi@newpi
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/Users/ryan/.ssh/id_rsa.github.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: WARNING: All keys were skipped because they already exist on the remote system.
		(if you think this is a mistake, you may want to use -f option)

OK … why is it sending the GitHub key? That’s a different problem for a different time. I see another flag available is the -i which will allow me to specify which key I want to send. Aha!

OK, now all that I need to do is use the following command to test the output:

ssh-copy-id -n -i ~/.ssh/id_rsa.pub pi@newpi

And sure enough it’s sending the correct key

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/Users/ryan/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: WARNING: All keys were skipped because they already exist on the remote system.
		(if you think this is a mistake, you may want to use -f option)

Remove the -n flag to send it for real

ssh-copy-id -i ~/.ssh/id_rsa.pub pi@newpi

And try to ssh in again

ssh pi@newpi

Success!

I wanted to write this up for 2 reasons:

  1. So I can refer back to it if I ever need to. This blog is mostly for me to write down technical things that I do so I can remember them later on
  2. This is the first time I’ve run into an issue with a command like tool and simply used the help to figure out how to fix the problem and I wanted to memorialize that. It felt forking awesome to do that.

Footnote: Yes … calling my new raspberry pi newpi in my hosts file is dumb. Yes, when I get my next new Raspberry Pi I will be wondering what to call it. YEs, I am going to try and remember to make the change before it happens so that I don’t end up with the next Pi being called newnewpi and the one after that being newnewnewpi

Receipts

Every month I set up a budget for my family so that we can track our spending and save money in the ways that we need to while still being able to enjoy life.

I have a couple of Siri Shortcuts that will take a picture and then put that picture into a folder in Dropbox. The reason that I have a couple of them is that one is for physical receipts that we got at a store and the other is for online purchases. I’m sure that these couple be combined into one, but I haven’t done that yet.

One of the great things about these shortcuts is that they will create the folder that the image will go into if it’s not there. For example, the first receipt of March 2019 will create a folder called March in the 2019 folder. If the 2019 folder wasn’t there, it would have created it too.

What it doesn’t do is create the sub folder that all of my processed receipts will go into. Each month I need to create a folder called month_name Processed. And each month I think, there must be a way I can automate this, but because it doesn’t really take that long I’ve never really done it.

Over the weekend I finally had the time to try and write it up and test it out. Nothing too fancy, but it does what I want it to do, and a little more.

# create the variables I'm going to need later

y=$( date +"%Y" )
m=$( date +"%B" )
p=$( date +"%B_Processed" )

# check to see if the Year folder exists and if it doesn't, create it
if [ ! -d /Users/ryan/Dropbox/Family/Financials/$y ]; then
	mkdir /Users/ryan/Dropbox/Family/Financials/$y
fi

# check to see if the Month folder exists and if it doesn't, create it
if [ ! -d /Users/ryan/Dropbox/Family/Financials/$y/$m ]; then
	mkdir /Users/ryan/Dropbox/Family/Financials/$y/$m
fi

#check to see if the Month_Processed folder exists and if it doesn't, create it
if [ ! -d "/Users/ryan/Dropbox/Family/Financials/$y/$m/$p" ]; then
	mkdir "/Users/ryan/Dropbox/Family/Financials/$y/$m/$p"
fi

The last section I use the double quotes “” around the directory name so that I can have a space in the name of the processed folder. Initially I had used an underscore but that’s not how I do it in real life when creating the sub directors, so I had to do a bit of googling and found a helpful resource.

The only thing left to do at this point is get it set up to run automatically so I don’t have to do anything.

In order to do that I needed to add the following to my cronjob:

0 5 1 * * /Users/ryan/Documents/scripts/create_monthly_expense_folders.sh

And now I will have my folder structure created for me automatically on the first of the month at 5am!