Building your own DVR Part II

This is a continuation from Raspberry Pi + HD Homerun Dual = OTA Dvr

The original post here was outdated and I decided to remove it.

There will be future posts but it's slow going.

This is going to be a very step by step tutorial as I go through things :)

Raspberry Pi + HDHomeRun Dual = OTA DVR

My latest project has involved working towards a cord cutting world.

Lets get started.

The Parts

(prices as of current posting)
35.09 Raspberry Pi Model B+ (I initially used, Raspberry Pi Model B but don't expect any difference)
About $10 Research your own microsd card (for Model B I used this, I will post what I use when I get my B+)
18.99 Powered USB Hub (Any version should work, I just picked something cheap)
110.77 HDHomeRun Dual (There is an upgrade the HDHomeRun Connect-2, I haven't tried it but if you're feeling luck here's the link HD Homerun Extend-2, looks like it'll encode to other formats which is nice).
45.74 HD Antenna (This one picks quite a few channels up and has worked really well thus far, Antennaweb.com recommended a different one, but after adding a mount it was more than this one).
47.99 Router (I'm using the WRT54G because it was a spare one I had on hand).No Longer Needed.
64.99 1 TB Harddrive (I had an existing one, so some of these costs I'm not accounting for myself so my break even is sooner)
This comes out to a grand total of about $300. In my case I pay around $28 per month for my Dish Network Subscription. This means it'll take to about 12 months and I'll be ahead for costs (assuming I buy everything at once, and don't have spare parts I can just use).

Note: I didn't include powering the Raspberry pi in this calculation of cost, I plan to see if I can plug directly into the powered usb hub, and plug in the HDD from there, but I don't know for sure on that, however it uses the same plug as MOST new phones out there these days, so you probably already have a cable. (one can be found here though)

Setup

The HD Home Run Dual is a REALLY Easy piece of equipment to use. All I did was plug it into the wall, plug into the router, and plug into the antenna.

You ABSOLUTELY need to be connected over ethernet with the raspberry pi (So NO raspberry pi Model A since it doesn't have an ethernet port).

I installed Rasbian (at some point in the past).

For the setup I booted into LXDE, (login and type startx).

I found this useful to remove a few unnecessary programs, but since we have a powered usb hub we are going to be storing the data on an external drive as the size of the saved file is a bit large.

http://www.sbprojects.com/projects/raspberrypi/tweaks.php

The Raspberry Pi will also need the following:

Download libhdhomerun, HDHomeRun Config GTK from here:
http://www.silicondust.com/support/downloads/linux/

Here is a GIST of the commands I used after I downloaded and extracted the above zip files, and I entered into the hdhomerun_config_gui folder.



Finally I ran the following to test that the tuner was running.

hdhomerun_config_gui

I then updated the firmware on the tuner (Disclaimer, do so at your own risk, not sure if it's required, but I did it).

Clicking on the Update tab, you can select the firmware also downloaded from Silicondust website above,

If you're using the DUAL it should be this driver: HDHR3-US (hdhomerun3_atsc).

If  you're using the Extend you'll have to research which version of the firmware is right.

Recording

At this point I believe I exited the gui and re-opened it once I got the firmware updated (I got worried for a split second cause it couldn't find my tuner anymore, but it was something as simple as that to get going again). I was able to scan and see some channels and click on View to see them playing.

If you note on the left side the name of the tuner is listed (I believe it's id - 0,1), you'll need this number, also if you type hdhomerun_config discover via command line you should be able to get it right.

My general process for recording TV is this: (may be improved in the future)

1. Set Channel
2. Save Stream for some period of time.
3. Open Handbrake and transcode to another format.

I decided to just save the default format of the stream (TS?).

Looking at the developer gui You will see there is the ability to pipe into another program.

As of right now I believe that the "save" command will basically just write bytes to a file. So there is very little processing overhead happening, adding transcoding into the mix might be too much for the pi (but may not be, I really haven't verified).

I haven't tried it but you MAY be able to record BOTH tuners from one raspberry pi at a time.

Here is the single line command I used to record something last night, I'll explain what each means in a minute.

date; sleep 3600; date; hdhomerun_config set /tuner0/channel auto:27; timeout 3700 hdhomerun_config save /tuner0 test_show.ts;

Ok so I'll break down each section.

date prints the date/time
sleep will sleep the number of seconds allocated, I put in the dates around to verify it is waiting the right amount of time, and it appears to be.
hdhomerun_config ID set /tuner0/channel auto:27; This will set the first tuner to channel 27 auto selecting the modulation
timeout 3700 run a command for just over an hour (When I ran the .ts file, it was usually a little shorter than the timeout time, so I just expand it over a bit, you can edit it back if you need to using video editing software).
hdhomerun_config save /tuner0 test_show.ts; This is what actually saves your OTA channel.

Note: Remember to navigate to your external hard drive when you run this (you may be able to point the .ts file to the path, but I haven't tried it), otherwise you'll run out of memory pretty fast. When I recorded about 4.5 hours it came out to 26.1 GB of data. Transcoding down should come out to 5-7 GB so it'll go quite a bit down (using Normal on Handbrake and MKV, other settings may come out smaller).

Good luck and let me know what you come up with!

Keep an eye out for a future post, I'm planning on building a NODEJS server that will allow me to schedule recordings from the internet this will be similar to my post about controlling my roku/chromecast from the internet, I'll be using Firebase as an intermediary. (I'd love to go full on DVR solution but just worry MythTV or some other equivalent would be too heavy).

Automated Email Form Part 2

Continuing our post from Automated Email Form Part 1.

Now you have a trigger setup in Zapier, lets setup the webpage to send an email to yourself.

This will be an AngularJS app, so add the appropriate pieces to make it work. (you can look at angularjs.org or many online tutorials for specifics).

First lets add an input field to your webpage like so:



If you notice I left name in there, but i'm not collecting names, I decided the less a user has to type the lower the barrier to contacting me in this case. Maybe one day I'll add a name if I feel its necessary, but just adding one more field with the NG model set to name would capture the users name for yourself.

Basically I setup an NG-Click to call a function in my angularjs code.

So while in some cases you could put your angularjs code in html, most of the time you put it in a .js file, (I'm putting in the .html so there are script tags surrounding, but generally don't do that, unless it's a REALLY simple app, which in this case it is).



If you notice I do a simple regex email validate check.

When it passes it pushes the email address to my firebase link, next Zapier picks up the change and sends you an email.

Pro Tip: In fact I almost forgot I had it setup and when I received an email with the submitted email, I got really confused. so DON'T forget to make the subject easily recognizable, otherwise you might run into the same case :)

Good luck and let me know how it works for you!

Automated Email Form Part 1

I'm kinda going to do this backwards, so hang with me a bit. First I'll talk about what my problem was that Zapier solved, and then show off the code that was in use.

So I have this nifty thing on my website (onaclovtech.com) that asks people to put in their email address. It then stores to Firebase. This is great except for one slight problem. I have to go check periodically if anyone added their email address.

If you actually do this it works well, however I don't always remember to (I really wasn't actually expecting any emails added for a while). Sorry to the 3 people who I discovered in fact DID use my form.

Ok, so I rambled for a bit, but really what I wanted to tell you was a way to solve your problem. Introducing Zapier. Zapier takes tasks and connects them together (very similar to IFTTT if you've ever played with it). I setup a Zap (what it's called when you connect two things together) to check when new child is added to an object in Firebase.

When a new child is added it emails me with the contacts email, and "new email submitted" in the subject line. This helps me keep on top of "new" visitors that want to chat.

 Head on over to Zapier if you're interested in automating this kind of stuff.
I made a template if you use it you'll get some extra tasks I believe.
 Heres the link: http://zpr.io/G6ri


 NOTE: By signing up with Zapier through one of my links I'll get some bonus tasks (A. La. Dropbox style), if you do sign up and want to see whatever other ones are around check this link out: https://zapier.com/rewards/ You've been warned :)


Stay tuned for part two!

UPDATE: Part 2 is Live here --> Automated Email Form Part 2

Control your Network Connected Devices

Today I want to tell you about a little epiphany I had a few weeks back. I realized that I could connect firebase to a local nodejs server and a public facing website, and control my roku.

Most people read this and think....well duh you can already control your roku using your phone. At this time (please prove me wrong) you need to be on your Intranet. Your Intranet is the network in your house that your roku is on, so basically you have to be on the same wifi as your roku.

This means you can setup a website using firebase to authenticate you, connect to a common data point in your database and now control things in your house from ANYWHERE as long as both have an internet connection. So I could be in Hong Kong and change the channel on my roku.

Ok, big deal why should I care? Well what if I told you that your roku isn't the only device that connects to the internet? You can query for all these devices, setup commands for them, and control them from anywhere! Forgot a light on in the house when you left? With (some) devices you can check the status and turn it off/on. This kinda makes that whole "internet" of things a lot closer to reality without completely opening up your network, the only "failure" point here is if your firebase gets hacked or something.

But you want code....

https://github.com/onaclovtech/MediaController

Here is the side that runs in your house (and you could also just run this on a raspberry pi or something instead of a dedicated full blown computer).

Right now I have my own rolled roku controller, but I think there is one in npm already that I haven't checked out. You can control your Chromecast this way. Possibly smart tv's etc.


Details for the people who are interested:

Node Side:
1. Start server
2. Server does a query for network devices and adds them to a list.
3. Server pushes list to myfirebaseurl/mediadevices
4. Server then sets up a callback to trigger when a child changes (state specifically).
5. Server sets up functions to handle deleting devices in mediadevices when it closes (this is to ensure that the devices are always accessible).

AngularJS side:
1. Connect to myfirebaseurl/mediadevices
2. Associate NG model to a dropdown (that's how I'm swapping between, if you have other ideas I'd love to hear them).
3. When a button is pressed, it looks at which device is selected and changes it's state to whatever mode that button is designed to accomplish.
4. If the server is removed then mediadevices is empty and the dropdown goes away.




Let me know what you think! If you have a chromecast, amazontv, etc that you want to build this on, or want to lend me to build it on, lemme know I'd love to chat!

Tyson

Movies Oh My

I've been a busy boy.

In my quest to generate a boatload of Angular Services (for angularjsservices.com, dont hate the site it is terrible, if you want to help build it let me know!).

I wrote up another library but this time for my movies.

https://github.com/onaclovtech/movielibrary

The cool thing here is I built a rottentomatoes.js angular service for it.

https://github.com/onaclovtech/rottentomatoes

I got it all nice looking and then discovered omdbapi and guess what? now we have one for that too.

https://github.com/onaclovtech/omdbapi

You can substitute them and get your data.

Now the bad news. (but potentially good news).

The JSON returned is different between the two. So something I'm thinking about that would be really slick is if we could essentially take the format from each and push the data to a title,picture,etc variables, then we simply return our search and it contains an object with the appropriate fields in a standard location. I think this would be huge for ALL of my Angular Services to basically build a JSON format that will be consistent, and the service will manage converting.

If you're interested in helping out with coming up with standards for this I would love it. If you're interested in helping build angularjsservices to what it SHOULD be, I would love that.

In the mean time I'm building building building.

Join me and build an angular service too? (trust me they're surprisingly easy, and they make using angular even more fun)

I'm using Firebase to store off the data just like my book library. Firebase sure is handy.

While playing with this I realized something....I can control my roku with a webpage and I don't need chrome's special stuff (that the chromecast is using) I can just use a local nodejs server and firebase and host my site anywhere. I will be working on that next. Can you figure out how to do it? I think I've given some good hints :)

Emailster

+

+

=

Emailster

Today I'm planning to talk about SendGrid, AngularJS and FireBase. SendGrid is an email delivery service which supports a REST based interface for sending emails to customers, FireBase manages JSON data and syncing that data between clients really fast. Finally AngularJS provides us with a client side framework that has great databinding and is pretty fast and easy to use.

My goal was to make a client side only app and send emails from it as well as determine the validity of those email addresses.

I created a service called email and it's located in a file called sendgrid.js.

Now in theory anyone who can manage sending emails via REST or other web technology can be inserted in place as long as the function prototypes match. In this case I have a function called "send" that takes the parameters of api key/user, who the message is going to, what the subject is, what the body will say, and who the message is from. I'm working on a site called angularjsservices.com which I hope to collect all these services we can interchange and use for creating amazing client side only applications.

I am marrying Firebase by keeping track of what emails are valid, AngularJS to manage routing for the Firebase "ID" the user is assigned and setting it's validity in Firebase.

The whole reason I came up with this idea was because sometimes you have an idea you want to see if there is interest for. So you want to collect valid emails, or you want to give something away if someone provides a valid email address, this can be accomplished using these client side only technologies.

In Firebase I create a data structure like this:

{
  email: jon@doe.com,
  name: Jon Doe,
  valid: false
}

Next I setup routing in angularjs

webpage.com/:id

where :id is the firebase data element.

So now we can do something like this

firebaseurl.com/:id

to point to the specific firebase entity.

When the user clicks on their link webpage.com/:id it sets the validity of the email to true:
it becomes this:

{
  email: jon@doe.com,
  name: Jon Doe,
  valid: true
}

And there you have it, you can collect emails from a client side only application.

API Key and User
When you make a REST call to SendGrid it's passes your API Key and User in plain text, when you have a server application it's not *nearly* as big of a deal, however when you have a client side ONLY application, this becomes a big problem since your API Key and User are the User/Password to SendGrid's website.
I have a few suggestions. The first is Stop using the same User/Password as your API key, and allow you to generate new ones if you happen to require changing yours. This would prevent people from running off with your keys and logging in and changing settings. It would also allow you to change it if someone decided to grab your API Key and user in your client side only app and start using them. The second thing that could happen is similar to FireBase you could enable security settings for requests from specific domains. This solves the API Key and User name in the client side only app because we'll only be able to "send" and email if you're ON that domain. Either way splitting the API Key and User from the login credentials is a critical issue for me (and if I'm wrong PLEASE PLEASE let me know and I'd be happy to update this post).

XMLHttpRequest
Second is that when you make a REST post using my Angular Service I get a XMLHttpRequest cannot load [sent REST message url]  The 'Access-Control-Allow-Origin' header has a value ... that is not equal to the supplied origin. Origin 'null' is therefore not allowed access. If you click on the link the value in the browser says this:
{"message":"success"}

Which I find very surprising since it appears I got an error, I'm not sure if this is a mistake on my side or theirs, but either way I am able to send a message successfully so somethings up here. Again please contact me if you're able to help out.

You can see the application as it is here:
https://github.com/onaclovtech/Emailster
I'll slowly be cleaning it up and making it fancier, but I really wanted to show it off right away!


What other apps can we make that traditionally require a server side component, but we can write as client side only now?

Saving user data without requiring login

Saving users data without requiring a login can be a tricky problem, but with firebase and a framework that supports routing it doesn't have to be.

I have a friend who has a site called http://www.infantchart.com/. He mentioned one time that people don't like to have to login, or link to a google account so they can save their data. I think we've done this so many times and we understand the reasoning why, that it was hard to understand why someone wouldn't want to do that.

I have been playing with Firebase a ton, and I love it. Just last night I wrote a VERY simple app that adds elements to my data store. I had nearly forgotten that when I add a JSON object to my store, it'll generate a unique id (like a primary key in a typical database). Having worked with Angular JS, I know you can use routing pretty easily. So I now have a solution for his problem.  When a user enters in some data, generate a "new" key for them, and redirect to that page for any further input. Now when the user visits infantchart.com/[unique id from firebase] using the ability to route, you simply open a firebase connection to [firebaseurl]/[uniqueid]

How do we get the user to go to that site though? They're probably not going to remember HvawoinvNOINRV to be able to get to their data. Adding a "share" option and really highlight it so they share with friends, this will attract new visitors, as well as help them find their way back. More than likely if they can't remember the path, they'll remember they shared it with friends and find the link that way. Additionally you can store that unique id into local storage so when they visit from the same device it automatically goes to the right link, this way they can always find their way back.

Giving users the ability to save their data without having to actually login/link will help retention and reuse, resulting in a more useful site and a little more intuitive I'd like to think.

Disclaimer: Don't use this approach for sensitive data since finding other peoples data is a matter of cycling through different combinations (it might be a pain and hard to find stuff, but it's not impossible).

Do you have other ways to save user data without requiring a login, love to hear about em in the comments.

Angular And Firebase Routing and Small Data Sets

I was learning how to use Firebase with Angular recently, and I ran into a problem, when I asked a question on stack overflow I received an answer basically saying to call firebaseurl/ of the data, well I can get the index, but I wasn't sure how my second controller could get that data ( so when my router swapped to the new page, I didn't know how to get it). Thanks to a generous soul at Open Hack last night, we discovered $routeParams, apparently when you use :id in your router config, when you use $routeParams in your "new" controller, you can access whatever that : was, so in this case $routeParams.id gave me the id, which is precisely what I needed. 

I just wanted to point this out in case anyone else had issues. So long story short,

First Page had this:
<a href="http://www.blogger.com/blogger.g?blogID=4424951240484003947#/blah/{{index}}">Blah </a>(Where index is my id)

the router config looked like this

path="/blah/:id"

the "new controller" added a reference to $routeParams (like where we reference firebase, just add a comma and go).
Then I did

"firebaseurl" + $routeParams.id to capture a smaller set of data.