That moment where you should have automated but didn't


  • Moderador

    TLDR; I migrated blogs over to Hugo and I didn’t automate because I was only doing it once, I should have automated because I actually migrated 450+ times (at least once per post. Find results at testerhq.com


    On Risk Aversion

    Because I’m slightly risk aware I have the following concerns about my main blogs:

    • blogger.com is free so at some point Google will cancel it and I will lose everything
    • wordpress uses php and a database and at somepoint someone will take advantage of a security hole and I’ll get hacked and lose everything

    When I post to linkedin I have the concern that:

    • there is no RSS feed
    • linkedin might go down and I’ll lose everything

    And if I ever start posting to medium or other platforms I will have the same issues.

    Risk Mitigation

    Therefore I decided to ‘archive’ everything into a static site so that if everything goes down I have a backup.

    I chose to use Hugo for this.

    Subtle Wins

    This has the advantage that:

    • all posts in Hugo are written in markdown
    • I already write all my posts in markdown (and covert into html for pasting into the other platforms)
    • I get to use my testerhq.com domain
    • I get to try out the multiple site hosting on my vidahost cloud plan
    • much harder to hack a static site (and if you do, I just re-upload)
    • I can eventually put all posts on github and have an offsite source back up as well
    • I can repurpose all the posts later

    I will use testerhq.com as my aggregator and I will probably extend it to cover:

    • stuff I post to slideshare
    • instagram
    • tweets I think are useful
    • youtube videos etc.

    But I didn’t Automate the migration

    I looked around for tools that could easily migrate from Wordpress to Hugo and blogger to Hugo but didn’t find a lot that fit my workflow.

    I thought I should really write my own but since I saw so many failed attempts out there I thought it might be too hard and take too long.

    • I really should have done

    I thought that since I’ll just migrate once, that I really couldn’t justify the time to automate it.

    • I might migrate each blog once (4 of them), but there are over 450 posts, that might be worth automating.

    I found a solution

    My final solution was:

    • Use wordpress blogger import
    • Use a jekyll export Wordpress plugin
    • import from Jekyll to Hugo

    I used a Wordpress bitnami VM and imported my blogger blogs to that, then exported to Jekyll and then imported to Hugo.

    I exported my wordpress blogs directly to Jekyll and imported to Hugo.

    And then the pain started.

    I should probably have automated this part

    I then had to check every post for mistakes during the various migration.

    Sometimes it was:

    •   unconverted
    • html for iframes of slideshare and youtube messed up
    • images not convered to markdown properly
    • mad html not converted properly due to a missing tag somewhere
    • various special chars not converted properly

    Pretty much every post needed to be hand checked and amended with find and replace in some form or other.

    It took me ages. I suspect it would have been faster for me to automate this last part.

    I did automate something

    I dug out my old HTTP Testing Web Crawler and amended it to check for characters on the page this made it easy for me to check for:

    • images that hadn’t converted properly e.g. and left ![]( type chars on the html

    I still have a lot of broken links to stuff I’ve linked to over the years, and I’ll slowly fix that up over time.

    I almost repeated the same mistake

    I was looking at an old Choose Your Own Adventure Game that I created a few years ago in Twine with my Son. And thought about converting it into more of a CYOA game book format e.g. walk out of the door (goto 8) listed on the page rather than a link.

    I started hand converting it from a twine archive into markdown because I looked around and could not find any tools available that did this, and I saw a few failed attempts and I was about to say ‘this is hard’.

    But I knocked up a quick converter using Java and JSoup and it didn’t take long at all.

    The benefit is, that because I have automated:

    • I am more likely to do it again
    • I will probably write more CYOA games
    • I learned more about JSoup
    • I have yet more code on github
    • I re-used the work I put into pandocifier

    Summary

    Leia mais em http://blog.eviltester.com/2017/04/input-process-output-i-test-systems.html