This is a continuation from Raspberry Pi + HD Homerun Dual = OTA Dvr
The original post here was outdated and I decided to remove it.
There will be future posts but it's slow going.
This is going to be a very step by step tutorial as I go through things :)
Tuesday, December 23, 2014
Monday, December 22, 2014
Raspberry Pi + HDHomeRun Dual = OTA DVR
My latest project has involved working towards a cord cutting world.
Lets get started.
The Parts
(prices as of current posting)
35.09 Raspberry Pi Model B+ (I initially used, Raspberry Pi Model B but don't expect any difference)
About $10 Research your own microsd card (for Model B I used this, I will post what I use when I get my B+)
18.99 Powered USB Hub (Any version should work, I just picked something cheap)
110.77 HDHomeRun Dual (There is an upgrade the HDHomeRun Connect-2, I haven't tried it but if you're feeling luck here's the link HD Homerun Extend-2, looks like it'll encode to other formats which is nice).
45.74 HD Antenna (This one picks quite a few channels up and has worked really well thus far, Antennaweb.com recommended a different one, but after adding a mount it was more than this one).
47.99 Router (I'm using the WRT54G because it was a spare one I had on hand).No Longer Needed.
64.99 1 TB Harddrive (I had an existing one, so some of these costs I'm not accounting for myself so my break even is sooner)
This comes out to a grand total of about $300. In my case I pay around $28 per month for my Dish Network Subscription. This means it'll take to about 12 months and I'll be ahead for costs (assuming I buy everything at once, and don't have spare parts I can just use).
Setup
The HD Home Run Dual is a REALLY Easy piece of equipment to use. All I did was plug it into the wall, plug into the router, and plug into the antenna.
You ABSOLUTELY need to be connected over ethernet with the raspberry pi (So NO raspberry pi Model A since it doesn't have an ethernet port).
I installed Rasbian (at some point in the past).
For the setup I booted into LXDE, (login and type startx).
I found this useful to remove a few unnecessary programs, but since we have a powered usb hub we are going to be storing the data on an external drive as the size of the saved file is a bit large.
http://www.sbprojects.com/projects/raspberrypi/tweaks.php
The Raspberry Pi will also need the following:
Download libhdhomerun, HDHomeRun Config GTK from here:
http://www.silicondust.com/support/downloads/linux/
Here is a GIST of the commands I used after I downloaded and extracted the above zip files, and I entered into the hdhomerun_config_gui folder.
Finally I ran the following to test that the tuner was running.
hdhomerun_config_gui
I then updated the firmware on the tuner (Disclaimer, do so at your own risk, not sure if it's required, but I did it).
Clicking on the Update tab, you can select the firmware also downloaded from Silicondust website above,
If you're using the DUAL it should be this driver: HDHR3-US (hdhomerun3_atsc).
If you're using the Extend you'll have to research which version of the firmware is right.
Recording
At this point I believe I exited the gui and re-opened it once I got the firmware updated (I got worried for a split second cause it couldn't find my tuner anymore, but it was something as simple as that to get going again). I was able to scan and see some channels and click on View to see them playing.
If you note on the left side the name of the tuner is listed (I believe it's id - 0,1), you'll need this number, also if you type hdhomerun_config discover via command line you should be able to get it right.
My general process for recording TV is this: (may be improved in the future)
1. Set Channel
2. Save Stream for some period of time.
3. Open Handbrake and transcode to another format.
I decided to just save the default format of the stream (TS?).
Looking at the developer gui You will see there is the ability to pipe into another program.
As of right now I believe that the "save" command will basically just write bytes to a file. So there is very little processing overhead happening, adding transcoding into the mix might be too much for the pi (but may not be, I really haven't verified).
I haven't tried it but you MAY be able to record BOTH tuners from one raspberry pi at a time.
Here is the single line command I used to record something last night, I'll explain what each means in a minute.
date; sleep 3600; date; hdhomerun_config set /tuner0/channel auto:27; timeout 3700 hdhomerun_config save /tuner0 test_show.ts;
Ok so I'll break down each section.
date prints the date/time
sleep will sleep the number of seconds allocated, I put in the dates around to verify it is waiting the right amount of time, and it appears to be.
hdhomerun_config ID set /tuner0/channel auto:27; This will set the first tuner to channel 27 auto selecting the modulation
timeout 3700 run a command for just over an hour (When I ran the .ts file, it was usually a little shorter than the timeout time, so I just expand it over a bit, you can edit it back if you need to using video editing software).
hdhomerun_config save /tuner0 test_show.ts; This is what actually saves your OTA channel.
Keep an eye out for a future post, I'm planning on building a NODEJS server that will allow me to schedule recordings from the internet this will be similar to my post about controlling my roku/chromecast from the internet, I'll be using Firebase as an intermediary. (I'd love to go full on DVR solution but just worry MythTV or some other equivalent would be too heavy).
Lets get started.
The Parts
(prices as of current posting)
35.09 Raspberry Pi Model B+ (I initially used, Raspberry Pi Model B but don't expect any difference)
About $10 Research your own microsd card (for Model B I used this, I will post what I use when I get my B+)
18.99 Powered USB Hub (Any version should work, I just picked something cheap)
110.77 HDHomeRun Dual (There is an upgrade the HDHomeRun Connect-2, I haven't tried it but if you're feeling luck here's the link HD Homerun Extend-2, looks like it'll encode to other formats which is nice).
45.74 HD Antenna (This one picks quite a few channels up and has worked really well thus far, Antennaweb.com recommended a different one, but after adding a mount it was more than this one).
64.99 1 TB Harddrive (I had an existing one, so some of these costs I'm not accounting for myself so my break even is sooner)
This comes out to a grand total of about $300. In my case I pay around $28 per month for my Dish Network Subscription. This means it'll take to about 12 months and I'll be ahead for costs (assuming I buy everything at once, and don't have spare parts I can just use).
Note: I didn't include powering the Raspberry pi in this calculation of cost, I plan to see if I can plug directly into the powered usb hub, and plug in the HDD from there, but I don't know for sure on that, however it uses the same plug as MOST new phones out there these days, so you probably already have a cable. (one can be found here though)
Setup
The HD Home Run Dual is a REALLY Easy piece of equipment to use. All I did was plug it into the wall, plug into the router, and plug into the antenna.
You ABSOLUTELY need to be connected over ethernet with the raspberry pi (So NO raspberry pi Model A since it doesn't have an ethernet port).
I installed Rasbian (at some point in the past).
For the setup I booted into LXDE, (login and type startx).
I found this useful to remove a few unnecessary programs, but since we have a powered usb hub we are going to be storing the data on an external drive as the size of the saved file is a bit large.
http://www.sbprojects.com/projects/raspberrypi/tweaks.php
The Raspberry Pi will also need the following:
Download libhdhomerun, HDHomeRun Config GTK from here:
http://www.silicondust.com/support/downloads/linux/
Here is a GIST of the commands I used after I downloaded and extracted the above zip files, and I entered into the hdhomerun_config_gui folder.
Finally I ran the following to test that the tuner was running.
hdhomerun_config_gui
I then updated the firmware on the tuner (Disclaimer, do so at your own risk, not sure if it's required, but I did it).
Clicking on the Update tab, you can select the firmware also downloaded from Silicondust website above,
If you're using the DUAL it should be this driver: HDHR3-US (hdhomerun3_atsc).
If you're using the Extend you'll have to research which version of the firmware is right.
Recording
At this point I believe I exited the gui and re-opened it once I got the firmware updated (I got worried for a split second cause it couldn't find my tuner anymore, but it was something as simple as that to get going again). I was able to scan and see some channels and click on View to see them playing.
If you note on the left side the name of the tuner is listed (I believe it's id - 0,1), you'll need this number, also if you type hdhomerun_config discover via command line you should be able to get it right.
My general process for recording TV is this: (may be improved in the future)
1. Set Channel
2. Save Stream for some period of time.
3. Open Handbrake and transcode to another format.
I decided to just save the default format of the stream (TS?).
Looking at the developer gui You will see there is the ability to pipe into another program.
As of right now I believe that the "save" command will basically just write bytes to a file. So there is very little processing overhead happening, adding transcoding into the mix might be too much for the pi (but may not be, I really haven't verified).
I haven't tried it but you MAY be able to record BOTH tuners from one raspberry pi at a time.
Here is the single line command I used to record something last night, I'll explain what each means in a minute.
date; sleep 3600; date; hdhomerun_config set /tuner0/channel auto:27; timeout 3700 hdhomerun_config save /tuner0 test_show.ts;
Ok so I'll break down each section.
date prints the date/time
sleep will sleep the number of seconds allocated, I put in the dates around to verify it is waiting the right amount of time, and it appears to be.
hdhomerun_config ID set /tuner0/channel auto:27; This will set the first tuner to channel 27 auto selecting the modulation
timeout 3700 run a command for just over an hour (When I ran the .ts file, it was usually a little shorter than the timeout time, so I just expand it over a bit, you can edit it back if you need to using video editing software).
hdhomerun_config save /tuner0 test_show.ts; This is what actually saves your OTA channel.
Note: Remember to navigate to your external hard drive when you run this (you may be able to point the .ts file to the path, but I haven't tried it), otherwise you'll run out of memory pretty fast. When I recorded about 4.5 hours it came out to 26.1 GB of data. Transcoding down should come out to 5-7 GB so it'll go quite a bit down (using Normal on Handbrake and MKV, other settings may come out smaller).Good luck and let me know what you come up with!
Keep an eye out for a future post, I'm planning on building a NODEJS server that will allow me to schedule recordings from the internet this will be similar to my post about controlling my roku/chromecast from the internet, I'll be using Firebase as an intermediary. (I'd love to go full on DVR solution but just worry MythTV or some other equivalent would be too heavy).
Tuesday, November 25, 2014
Automated Email Form Part 2
Continuing our post from Automated Email Form Part 1.
Now you have a trigger setup in Zapier, lets setup the webpage to send an email to yourself.
First lets add an input field to your webpage like so:
If you notice I left name in there, but i'm not collecting names, I decided the less a user has to type the lower the barrier to contacting me in this case. Maybe one day I'll add a name if I feel its necessary, but just adding one more field with the NG model set to name would capture the users name for yourself.
Basically I setup an NG-Click to call a function in my angularjs code.
So while in some cases you could put your angularjs code in html, most of the time you put it in a .js file, (I'm putting in the .html so there are script tags surrounding, but generally don't do that, unless it's a REALLY simple app, which in this case it is).
If you notice I do a simple regex email validate check.
When it passes it pushes the email address to my firebase link, next Zapier picks up the change and sends you an email.
Pro Tip: In fact I almost forgot I had it setup and when I received an email with the submitted email, I got really confused. so DON'T forget to make the subject easily recognizable, otherwise you might run into the same case :)
Good luck and let me know how it works for you!
Now you have a trigger setup in Zapier, lets setup the webpage to send an email to yourself.
This will be an AngularJS app, so add the appropriate pieces to make it work. (you can look at angularjs.org or many online tutorials for specifics).
First lets add an input field to your webpage like so:
If you notice I left name in there, but i'm not collecting names, I decided the less a user has to type the lower the barrier to contacting me in this case. Maybe one day I'll add a name if I feel its necessary, but just adding one more field with the NG model set to name would capture the users name for yourself.
Basically I setup an NG-Click to call a function in my angularjs code.
So while in some cases you could put your angularjs code in html, most of the time you put it in a .js file, (I'm putting in the .html so there are script tags surrounding, but generally don't do that, unless it's a REALLY simple app, which in this case it is).
If you notice I do a simple regex email validate check.
When it passes it pushes the email address to my firebase link, next Zapier picks up the change and sends you an email.
Pro Tip: In fact I almost forgot I had it setup and when I received an email with the submitted email, I got really confused. so DON'T forget to make the subject easily recognizable, otherwise you might run into the same case :)
Good luck and let me know how it works for you!
Tuesday, September 23, 2014
100 year glasses
My parents crack me up. Anytime there's an argument, or concern about something they like to use the phrase
I always kinda brushed it off, and found it was useful to tell to others who were upset but not for yourself.
I was thinking about it just now and realized what if we started looking at things with 100 year glasses.
What would change, what things would matter?
I agree with Elon Musk (and I'm sure plenty of others) that becoming a multiplanetary race is a good thing.
I agree reducing our reliance on oil is a good thing (however I feel that reducing our need on energy in general is a good idea).
I agree finding a way to diagnose and resolve health issues without the need of doctors from your own home is necessary.
If resolving transportation issues is a need we as a people can and should solve it.
We should focus on furthering the human race more than helping a company making 100 billion dollars.
How can we still maintain a level of our fellow man while automating the unnecessary things? (some kind of robot to cook healthy meals for us in our own home)
If we can imagine it, and focus on it. We can build it.
Not everything should come at a cost.
What can you imagine we need built 100 years in the future with the will of our people?
Don't worry about it if it won't matter in 100 years
I always kinda brushed it off, and found it was useful to tell to others who were upset but not for yourself.
I was thinking about it just now and realized what if we started looking at things with 100 year glasses.
What would change, what things would matter?
I agree with Elon Musk (and I'm sure plenty of others) that becoming a multiplanetary race is a good thing.
I agree reducing our reliance on oil is a good thing (however I feel that reducing our need on energy in general is a good idea).
I agree finding a way to diagnose and resolve health issues without the need of doctors from your own home is necessary.
If resolving transportation issues is a need we as a people can and should solve it.
We should focus on furthering the human race more than helping a company making 100 billion dollars.
How can we still maintain a level of our fellow man while automating the unnecessary things? (some kind of robot to cook healthy meals for us in our own home)
If we can imagine it, and focus on it. We can build it.
Not everything should come at a cost.
What can you imagine we need built 100 years in the future with the will of our people?
Automated Email Form Part 1
So I have this nifty thing on my website (onaclovtech.com) that asks people to put in their email address. It then stores to Firebase. This is great except for one slight problem. I have to go check periodically if anyone added their email address.
If you actually do this it works well, however I don't always remember to (I really wasn't actually expecting any emails added for a while). Sorry to the 3 people who I discovered in fact DID use my form.
Ok, so I rambled for a bit, but really what I wanted to tell you was a way to solve your problem. Introducing Zapier. Zapier takes tasks and connects them together (very similar to IFTTT if you've ever played with it). I setup a Zap (what it's called when you connect two things together) to check when new child is added to an object in Firebase.
When a new child is added it emails me with the contacts email, and "new email submitted" in the subject line. This helps me keep on top of "new" visitors that want to chat.
Head on over to Zapier if you're interested in automating this kind of stuff.
I made a template if you use it you'll get some extra tasks I believe.
Heres the link: http://zpr.io/G6ri
NOTE: By signing up with Zapier through one of my links I'll get some bonus tasks (A. La. Dropbox style), if you do sign up and want to see whatever other ones are around check this link out: https://zapier.com/rewards/ You've been warned :)
Stay tuned for part two!
UPDATE: Part 2 is Live here --> Automated Email Form Part 2
Monday, September 22, 2014
Question Everything
Always ask why. Don't take "this is the way we've always done it" as the answer.
Take charge of your destiny. If something doesn't make sense. Don't just ask why, question everything about it.
Right now I'm working on the side trying to simplify our build process.
"Why do we need these buttons" ... "because we need to make all these variations of selections"
(After looking a little closer, the variations we selected happened pretty consistently, so guess what? a few checkboxes replace a bunch of radio buttons and other selections).
"Why do we let them select this data" .... "Because they might want to select something else"
(rarely if ever has anyone ever selected anything else)
Keeping on doing the same things because that's the way we've always done them, doesn't make your job easier, it makes it harder. We're software engineers it's in our blood to make complicated things simple and simple things complicated.
Good luck and godspeed!
Take charge of your destiny. If something doesn't make sense. Don't just ask why, question everything about it.
Right now I'm working on the side trying to simplify our build process.
"Why do we need these buttons" ... "because we need to make all these variations of selections"
(After looking a little closer, the variations we selected happened pretty consistently, so guess what? a few checkboxes replace a bunch of radio buttons and other selections).
"Why do we let them select this data" .... "Because they might want to select something else"
(rarely if ever has anyone ever selected anything else)
Keeping on doing the same things because that's the way we've always done them, doesn't make your job easier, it makes it harder. We're software engineers it's in our blood to make complicated things simple and simple things complicated.
Good luck and godspeed!
Labels:
inspiration
Thursday, August 28, 2014
Post JSON to Nodejs from Angularjs
I'm struggling with the title of this post as I searched many ways but couldn't find something simple like this.
If you have any other name suggestions I'm totally open!
Here goes.
I wanted to send JSON from an AngularJS app and get it on my NodeJS side so I could do something with the data.
There may be better ways (Socket.IO anyone?) but here goes:
Here is my angularJS service:
And here is my nodejs page:
Install express, and body-parser (I think that's about all) and you should be able to send a message using
$message.create({ }); // That's a JSON object being passed in
Simple AngularJS controller
And Finally a simple HTML page for it.
Good Luck!
I hope that helps SOMEONE out there.
If you have any other name suggestions I'm totally open!
Here goes.
I wanted to send JSON from an AngularJS app and get it on my NodeJS side so I could do something with the data.
There may be better ways (Socket.IO anyone?) but here goes:
Here is my angularJS service:
And here is my nodejs page:
Install express, and body-parser (I think that's about all) and you should be able to send a message using
$message.create({ }); // That's a JSON object being passed in
Simple AngularJS controller
And Finally a simple HTML page for it.
Good Luck!
I hope that helps SOMEONE out there.
Labels:
angularjs,
angularjs services,
json,
NodeJS
Wednesday, August 20, 2014
Control your Network Connected Devices
Today I want to tell you about a little epiphany I had a few weeks back. I realized that I could connect firebase to a local nodejs server and a public facing website, and control my roku.
Most people read this and think....well duh you can already control your roku using your phone. At this time (please prove me wrong) you need to be on your Intranet. Your Intranet is the network in your house that your roku is on, so basically you have to be on the same wifi as your roku.
This means you can setup a website using firebase to authenticate you, connect to a common data point in your database and now control things in your house from ANYWHERE as long as both have an internet connection. So I could be in Hong Kong and change the channel on my roku.
Ok, big deal why should I care? Well what if I told you that your roku isn't the only device that connects to the internet? You can query for all these devices, setup commands for them, and control them from anywhere! Forgot a light on in the house when you left? With (some) devices you can check the status and turn it off/on. This kinda makes that whole "internet" of things a lot closer to reality without completely opening up your network, the only "failure" point here is if your firebase gets hacked or something.
But you want code....
Here is the side that runs in your house (and you could also just run this on a raspberry pi or something instead of a dedicated full blown computer).
Right now I have my own rolled roku controller, but I think there is one in npm already that I haven't checked out. You can control your Chromecast this way. Possibly smart tv's etc.
Details for the people who are interested:
Node Side:
1. Start server
2. Server does a query for network devices and adds them to a list.
3. Server pushes list to myfirebaseurl/mediadevices
4. Server then sets up a callback to trigger when a child changes (state specifically).
5. Server sets up functions to handle deleting devices in mediadevices when it closes (this is to ensure that the devices are always accessible).
AngularJS side:
1. Connect to myfirebaseurl/mediadevices
2. Associate NG model to a dropdown (that's how I'm swapping between, if you have other ideas I'd love to hear them).
3. When a button is pressed, it looks at which device is selected and changes it's state to whatever mode that button is designed to accomplish.
4. If the server is removed then mediadevices is empty and the dropdown goes away.
Let me know what you think! If you have a chromecast, amazontv, etc that you want to build this on, or want to lend me to build it on, lemme know I'd love to chat!
Tyson
Most people read this and think....well duh you can already control your roku using your phone. At this time (please prove me wrong) you need to be on your Intranet. Your Intranet is the network in your house that your roku is on, so basically you have to be on the same wifi as your roku.
This means you can setup a website using firebase to authenticate you, connect to a common data point in your database and now control things in your house from ANYWHERE as long as both have an internet connection. So I could be in Hong Kong and change the channel on my roku.
Ok, big deal why should I care? Well what if I told you that your roku isn't the only device that connects to the internet? You can query for all these devices, setup commands for them, and control them from anywhere! Forgot a light on in the house when you left? With (some) devices you can check the status and turn it off/on. This kinda makes that whole "internet" of things a lot closer to reality without completely opening up your network, the only "failure" point here is if your firebase gets hacked or something.
But you want code....
https://github.com/onaclovtech/MediaController
Here is the side that runs in your house (and you could also just run this on a raspberry pi or something instead of a dedicated full blown computer).
Right now I have my own rolled roku controller, but I think there is one in npm already that I haven't checked out. You can control your Chromecast this way. Possibly smart tv's etc.
Details for the people who are interested:
Node Side:
1. Start server
2. Server does a query for network devices and adds them to a list.
3. Server pushes list to myfirebaseurl/mediadevices
4. Server then sets up a callback to trigger when a child changes (state specifically).
5. Server sets up functions to handle deleting devices in mediadevices when it closes (this is to ensure that the devices are always accessible).
AngularJS side:
1. Connect to myfirebaseurl/mediadevices
2. Associate NG model to a dropdown (that's how I'm swapping between, if you have other ideas I'd love to hear them).
3. When a button is pressed, it looks at which device is selected and changes it's state to whatever mode that button is designed to accomplish.
4. If the server is removed then mediadevices is empty and the dropdown goes away.
Let me know what you think! If you have a chromecast, amazontv, etc that you want to build this on, or want to lend me to build it on, lemme know I'd love to chat!
Tyson
Friday, August 15, 2014
Can we make web apps less reliant on network?
So I was thinking a little about how I was using bootstrap for some of my sites, and with it being a pretty commonly used code across sites it got me thinking.
Could we somehow share these resources across domains reducing the need to download, which causes us to slow things down normally.
Here is my thought.
When parsing a .html file you come across a script tag with a src a la
When it downloads the file it (maybe?) puts it in memory so it can be used (I could be wrong here).
What I was thinking is that we maybe generate a key value store, and it does something like this:
{"https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.2.6/angular.min.js", "pointer to memory that this resides"}
Now instead of simply downloading a .js file, it looks at the lookup (or hash table) and finds that that EXACT path file exists in memory and just uses that instead of downloading the file again.
Caveats.
1. I don't know if it would be possible to modify that memory.
2. I wouldn't expect this caching to be indefinite, but should be do-able across pages during an active session.
As long as the downloaded data isn't capable of being modified in memory, then doing a string lookup in a local hash *should* give us a decent startup decrease because we don't need to re-download the file(s) and I can't see how we would have any problems with safety/security since you're literally using the exact same file as you downloaded earlier, you just don't need to re-download.
I'm sure I'm missing something here, but I would think that this should be possible. Got any ideas? hit me up on twitter (@onaclov2000) or leave a comment here
Could we somehow share these resources across domains reducing the need to download, which causes us to slow things down normally.
Here is my thought.
When parsing a .html file you come across a script tag with a src a la
script src="https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.2.6/angular.min.js"
When it downloads the file it (maybe?) puts it in memory so it can be used (I could be wrong here).
What I was thinking is that we maybe generate a key value store, and it does something like this:
{"https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.2.6/angular.min.js", "pointer to memory that this resides"}
Now instead of simply downloading a .js file, it looks at the lookup (or hash table) and finds that that EXACT path file exists in memory and just uses that instead of downloading the file again.
Caveats.
1. I don't know if it would be possible to modify that memory.
2. I wouldn't expect this caching to be indefinite, but should be do-able across pages during an active session.
As long as the downloaded data isn't capable of being modified in memory, then doing a string lookup in a local hash *should* give us a decent startup decrease because we don't need to re-download the file(s) and I can't see how we would have any problems with safety/security since you're literally using the exact same file as you downloaded earlier, you just don't need to re-download.
I'm sure I'm missing something here, but I would think that this should be possible. Got any ideas? hit me up on twitter (@onaclov2000) or leave a comment here
Labels:
Browser
Wednesday, August 6, 2014
Movies Oh My
I've been a busy boy.
In my quest to generate a boatload of Angular Services (for angularjsservices.com, dont hate the site it is terrible, if you want to help build it let me know!).
I wrote up another library but this time for my movies.
https://github.com/onaclovtech/movielibrary
The cool thing here is I built a rottentomatoes.js angular service for it.
https://github.com/onaclovtech/rottentomatoes
I got it all nice looking and then discovered omdbapi and guess what? now we have one for that too.
https://github.com/onaclovtech/omdbapi
You can substitute them and get your data.
Now the bad news. (but potentially good news).
The JSON returned is different between the two. So something I'm thinking about that would be really slick is if we could essentially take the format from each and push the data to a title,picture,etc variables, then we simply return our search and it contains an object with the appropriate fields in a standard location. I think this would be huge for ALL of my Angular Services to basically build a JSON format that will be consistent, and the service will manage converting.
If you're interested in helping out with coming up with standards for this I would love it. If you're interested in helping build angularjsservices to what it SHOULD be, I would love that.
In the mean time I'm building building building.
Join me and build an angular service too? (trust me they're surprisingly easy, and they make using angular even more fun)
I'm using Firebase to store off the data just like my book library. Firebase sure is handy.
While playing with this I realized something....I can control my roku with a webpage and I don't need chrome's special stuff (that the chromecast is using) I can just use a local nodejs server and firebase and host my site anywhere. I will be working on that next. Can you figure out how to do it? I think I've given some good hints :)
In my quest to generate a boatload of Angular Services (for angularjsservices.com, dont hate the site it is terrible, if you want to help build it let me know!).
I wrote up another library but this time for my movies.
https://github.com/onaclovtech/movielibrary
The cool thing here is I built a rottentomatoes.js angular service for it.
https://github.com/onaclovtech/rottentomatoes
I got it all nice looking and then discovered omdbapi and guess what? now we have one for that too.
https://github.com/onaclovtech/omdbapi
You can substitute them and get your data.
Now the bad news. (but potentially good news).
The JSON returned is different between the two. So something I'm thinking about that would be really slick is if we could essentially take the format from each and push the data to a title,picture,etc variables, then we simply return our search and it contains an object with the appropriate fields in a standard location. I think this would be huge for ALL of my Angular Services to basically build a JSON format that will be consistent, and the service will manage converting.
If you're interested in helping out with coming up with standards for this I would love it. If you're interested in helping build angularjsservices to what it SHOULD be, I would love that.
In the mean time I'm building building building.
Join me and build an angular service too? (trust me they're surprisingly easy, and they make using angular even more fun)
I'm using Firebase to store off the data just like my book library. Firebase sure is handy.
While playing with this I realized something....I can control my roku with a webpage and I don't need chrome's special stuff (that the chromecast is using) I can just use a local nodejs server and firebase and host my site anywhere. I will be working on that next. Can you figure out how to do it? I think I've given some good hints :)
Tuesday, July 1, 2014
Emailster
+
+
=
Emailster
Today I'm planning to talk about SendGrid, AngularJS and FireBase. SendGrid is an email delivery service which supports a REST based interface for sending emails to customers, FireBase manages JSON data and syncing that data between clients really fast. Finally AngularJS provides us with a client side framework that has great databinding and is pretty fast and easy to use.
My goal was to make a client side only app and send emails from it as well as determine the validity of those email addresses.
I created a service called email and it's located in a file called sendgrid.js.
Now in theory anyone who can manage sending emails via REST or other web technology can be inserted in place as long as the function prototypes match. In this case I have a function called "send" that takes the parameters of api key/user, who the message is going to, what the subject is, what the body will say, and who the message is from. I'm working on a site called angularjsservices.com which I hope to collect all these services we can interchange and use for creating amazing client side only applications.
I am marrying Firebase by keeping track of what emails are valid, AngularJS to manage routing for the Firebase "ID" the user is assigned and setting it's validity in Firebase.
The whole reason I came up with this idea was because sometimes you have an idea you want to see if there is interest for. So you want to collect valid emails, or you want to give something away if someone provides a valid email address, this can be accomplished using these client side only technologies.
In Firebase I create a data structure like this:
{
email: jon@doe.com,
name: Jon Doe,
valid: false
}
Next I setup routing in angularjs
webpage.com/:id
where :id is the firebase data element.
So now we can do something like this
firebaseurl.com/:id
to point to the specific firebase entity.
When the user clicks on their link webpage.com/:id it sets the validity of the email to true:
it becomes this:
{And there you have it, you can collect emails from a client side only application.
email: jon@doe.com,
name: Jon Doe,
valid: true
}
API Key and User
When you make a REST call to SendGrid it's passes your API Key and User in plain text, when you have a server application it's not *nearly* as big of a deal, however when you have a client side ONLY application, this becomes a big problem since your API Key and User are the User/Password to SendGrid's website.
I have a few suggestions. The first is Stop using the same User/Password as your API key, and allow you to generate new ones if you happen to require changing yours. This would prevent people from running off with your keys and logging in and changing settings. It would also allow you to change it if someone decided to grab your API Key and user in your client side only app and start using them. The second thing that could happen is similar to FireBase you could enable security settings for requests from specific domains. This solves the API Key and User name in the client side only app because we'll only be able to "send" and email if you're ON that domain. Either way splitting the API Key and User from the login credentials is a critical issue for me (and if I'm wrong PLEASE PLEASE let me know and I'd be happy to update this post).
XMLHttpRequest
Second is that when you make a REST post using my Angular Service I get a XMLHttpRequest cannot load [sent REST message url] The 'Access-Control-Allow-Origin' header has a value ... that is not equal to the supplied origin. Origin 'null' is therefore not allowed access. If you click on the link the value in the browser says this:
{"message":"success"}
Which I find very surprising since it appears I got an error, I'm not sure if this is a mistake on my side or theirs, but either way I am able to send a message successfully so somethings up here. Again please contact me if you're able to help out.
You can see the application as it is here:
https://github.com/onaclovtech/Emailster
I'll slowly be cleaning it up and making it fancier, but I really wanted to show it off right away!
What other apps can we make that traditionally require a server side component, but we can write as client side only now?
Friday, June 27, 2014
Depth First vs Breadth First programming and stubbing
Originally I wanted to title this saying TDD instead of stubbing, but really the reason this topic came up is due to chatting with a friend about writing dummy functions to get TDD to pass, which lead to me commenting that sometimes I want to stub something out so I can say "well let's pretend that works" and move on and get the code I'm actively working, on working.
Breadth First Programming Style
This style of programming is writing the highest level (user facing we'll call it) code first, any calls to functions that should do something or get data or whatever should be quickly written as stubs with something to get you moving in your current function. The next step is moving into the first (or last depending on the order you'd like to go) and managing that functionality. If that calls down into nested functions then you follow the same organization, finish all the functions at the n-1 level of your earlier function and go on.
Depth First Programming Style
This style of programming is writing code that always works at each step. If you start writing and you come across a function you need, you simply drop down and implement it, and if that has functions it calls, you do the same. This keeps happening until you've finished everything down to the bottom, when you move up you do the same thing on the next function until you finally reach the top level and move along again.
Mixed Style
I like to work in a Mixed style. When you're working a complicated algorithm on your main thread of work it makes sense to say, well lets assume we get this data correctly and move on and continue working the algorithm, however if it's a relatively easy problem to solve and dropping into the function the function is a bear, it might make more sense to just implement that function first then move back up to your higher level.
TDD Aside
Since I brought it up I should talk a little about TDD. TDD is to me more of a Depth First style of programming. You write up a test, then you implement a high level of it, and yay it passes, then you write another test and drop back down to the function level you were at and "fix it" and keep doing that. I almost would compare this to a Yo-Yo strategy, you start at the top with a test, and throw your Yo-Yo, then when it gets all the way through the pass you start again at the top. You don't move forward until things are as right as you can make them.
How do you work what is your style? What do you think TDD is like? Are there any other programming styles you can think of in this context?
Breadth First Programming Style
This style of programming is writing the highest level (user facing we'll call it) code first, any calls to functions that should do something or get data or whatever should be quickly written as stubs with something to get you moving in your current function. The next step is moving into the first (or last depending on the order you'd like to go) and managing that functionality. If that calls down into nested functions then you follow the same organization, finish all the functions at the n-1 level of your earlier function and go on.
Depth First Programming Style
This style of programming is writing code that always works at each step. If you start writing and you come across a function you need, you simply drop down and implement it, and if that has functions it calls, you do the same. This keeps happening until you've finished everything down to the bottom, when you move up you do the same thing on the next function until you finally reach the top level and move along again.
Mixed Style
I like to work in a Mixed style. When you're working a complicated algorithm on your main thread of work it makes sense to say, well lets assume we get this data correctly and move on and continue working the algorithm, however if it's a relatively easy problem to solve and dropping into the function the function is a bear, it might make more sense to just implement that function first then move back up to your higher level.
TDD Aside
Since I brought it up I should talk a little about TDD. TDD is to me more of a Depth First style of programming. You write up a test, then you implement a high level of it, and yay it passes, then you write another test and drop back down to the function level you were at and "fix it" and keep doing that. I almost would compare this to a Yo-Yo strategy, you start at the top with a test, and throw your Yo-Yo, then when it gets all the way through the pass you start again at the top. You don't move forward until things are as right as you can make them.
How do you work what is your style? What do you think TDD is like? Are there any other programming styles you can think of in this context?
Labels:
Breadth First,
Depth First,
TDD
Wednesday, June 25, 2014
Ordered Entry of Form Data UX
As a follow up to this post: Stacking Enabled/Disabled States
I am going to show you a working example.
So now when combobox selected index changes it disables and clears all the form fields AFTER textbox1 (so textbox2, and button1)
Additionally you could set up a similar situation for adding data to comboboxes. If you enter in some data and move to the next field, you could call the same thing and have function pointers for the data on the "next" item in the list (so instead of pointing to 2 down you point to 1 down the list).
Good luck and do you have any tips for Ordered Entry of Form data?
I am going to show you a working example.
So now when combobox selected index changes it disables and clears all the form fields AFTER textbox1 (so textbox2, and button1)
Additionally you could set up a similar situation for adding data to comboboxes. If you enter in some data and move to the next field, you could call the same thing and have function pointers for the data on the "next" item in the list (so instead of pointing to 2 down you point to 1 down the list).
Good luck and do you have any tips for Ordered Entry of Form data?
Thursday, June 19, 2014
Stacking Enabled/Disabled States
I get annoyed when working with forms that require data to be entered in steps, this is usually how the code works.
1. Write a "on changed" event handler for the field.
2. When fired Set all fields below to disabled.
3. Enable only the next field.
4. Repeat.
All I can think about here is DRY (Don't Repeat Yourself) but how can we accomplish it?
I'm thinking you build an ordered list of all the fields, then when an on change event handler fires, it finds where your field is in the list, and disables all below it. So from there you simply make ONE function call in ALL cases, and pass it your current field, it'll then follow the rules.
What do you think? How do you manage ordered data entry and how you select appropriate fields?
I'll try implementing this and when I finish I will post here what my results were (pains/gains).
This is a rough idea, (not an actual implementation, but should be very close)
1. Write a "on changed" event handler for the field.
2. When fired Set all fields below to disabled.
3. Enable only the next field.
4. Repeat.
All I can think about here is DRY (Don't Repeat Yourself) but how can we accomplish it?
I'm thinking you build an ordered list of all the fields, then when an on change event handler fires, it finds where your field is in the list, and disables all below it. So from there you simply make ONE function call in ALL cases, and pass it your current field, it'll then follow the rules.
What do you think? How do you manage ordered data entry and how you select appropriate fields?
I'll try implementing this and when I finish I will post here what my results were (pains/gains).
This is a rough idea, (not an actual implementation, but should be very close)
Tuesday, June 3, 2014
Using zip on the command line
I was trying to find zip on the command line, I don't know all the parameters that are available. However, a little guessing I added -r but anywho, this is the command.
That's it.
Good luck
zip -r filename.zip foldername
That's it.
Good luck
Monday, May 19, 2014
How Did I Get Here
I blogged for work.....that's crazy right?
Check it out.
http://aerospace.honeywell.com/blog/how-did-i-get-here-my-path-to-honeywell
Check it out.
http://aerospace.honeywell.com/blog/how-did-i-get-here-my-path-to-honeywell
Wednesday, May 14, 2014
Saving user data without requiring login
Saving users data without requiring a login can be a tricky problem, but with firebase and a framework that supports routing it doesn't have to be.
I have a friend who has a site called http://www.infantchart.com/. He mentioned one time that people don't like to have to login, or link to a google account so they can save their data. I think we've done this so many times and we understand the reasoning why, that it was hard to understand why someone wouldn't want to do that.
I have been playing with Firebase a ton, and I love it. Just last night I wrote a VERY simple app that adds elements to my data store. I had nearly forgotten that when I add a JSON object to my store, it'll generate a unique id (like a primary key in a typical database). Having worked with Angular JS, I know you can use routing pretty easily. So I now have a solution for his problem. When a user enters in some data, generate a "new" key for them, and redirect to that page for any further input. Now when the user visits infantchart.com/[unique id from firebase] using the ability to route, you simply open a firebase connection to [firebaseurl]/[uniqueid]
How do we get the user to go to that site though? They're probably not going to remember HvawoinvNOINRV to be able to get to their data. Adding a "share" option and really highlight it so they share with friends, this will attract new visitors, as well as help them find their way back. More than likely if they can't remember the path, they'll remember they shared it with friends and find the link that way. Additionally you can store that unique id into local storage so when they visit from the same device it automatically goes to the right link, this way they can always find their way back.
Giving users the ability to save their data without having to actually login/link will help retention and reuse, resulting in a more useful site and a little more intuitive I'd like to think.
Disclaimer: Don't use this approach for sensitive data since finding other peoples data is a matter of cycling through different combinations (it might be a pain and hard to find stuff, but it's not impossible).
Do you have other ways to save user data without requiring a login, love to hear about em in the comments.
I have a friend who has a site called http://www.infantchart.com/. He mentioned one time that people don't like to have to login, or link to a google account so they can save their data. I think we've done this so many times and we understand the reasoning why, that it was hard to understand why someone wouldn't want to do that.
I have been playing with Firebase a ton, and I love it. Just last night I wrote a VERY simple app that adds elements to my data store. I had nearly forgotten that when I add a JSON object to my store, it'll generate a unique id (like a primary key in a typical database). Having worked with Angular JS, I know you can use routing pretty easily. So I now have a solution for his problem. When a user enters in some data, generate a "new" key for them, and redirect to that page for any further input. Now when the user visits infantchart.com/[unique id from firebase] using the ability to route, you simply open a firebase connection to [firebaseurl]/[uniqueid]
How do we get the user to go to that site though? They're probably not going to remember HvawoinvNOINRV to be able to get to their data. Adding a "share" option and really highlight it so they share with friends, this will attract new visitors, as well as help them find their way back. More than likely if they can't remember the path, they'll remember they shared it with friends and find the link that way. Additionally you can store that unique id into local storage so when they visit from the same device it automatically goes to the right link, this way they can always find their way back.
Giving users the ability to save their data without having to actually login/link will help retention and reuse, resulting in a more useful site and a little more intuitive I'd like to think.
Disclaimer: Don't use this approach for sensitive data since finding other peoples data is a matter of cycling through different combinations (it might be a pain and hard to find stuff, but it's not impossible).
Do you have other ways to save user data without requiring a login, love to hear about em in the comments.
Monday, May 5, 2014
The Passionate Programmer
Wednesday, April 2, 2014
How do we stop loved ones from clicking on spam/phishing links?
Do not let them ever get
the links.
Problem
Ok I am just being a
little facetious. I have kind of grown up on the internet, I am suspicious of
all kinds of things (and I am sure I make mistakes all the time), but there are
some cases that are just so obvious. However, here is the problem I have,
things that are obvious to me, are not obvious to everyone.
Is there a reason that
in the history of your email, you have NEVER sent a link to Botox to your
friends, in fact, you have NEVER sent an email to ALL your friends, nor have
you sent the same email (or close enough) to EACH of your friends in different emails?
I am willing to bet you have not sent the same email (or similar) to more than
10 friends in your contacts which falls into the HUNDREDs or possibly even
THOUSANDS.
Solutions
Send filter
Google reads your inbox,
Yahoo reads your inbox, and the NSA reads your inbox. I suspect they read your
outbox too. Why not read them for good too.
What I propose is that
the companies who are sending the email do a simple filter. IF we are trying to
send links to a BUNCH of people, OR multiple emails of the same link to a bunch
of people, we just simply hold onto the email, and wait a few days. Simply
popping up a notification probably will not help, since the person who hacked
the account would just as likely hit "OK". However, if you leave it
in your sent box with a reminder for say 3 days (this should be enough as MOST
people check their email at least 1 time in 3 days), that says "Hey you
sent this email to a bunch of people x days ago, I can still cancel it if you'd
like".
Alternately, we could
add some kind of are you a real person test (CAPTCHA), so now instead of
sending 100 emails all at once, (or say more than 3) you have to put in a CAPTCHA
for every so many, which would slow down anyone sending spam. Make it a poor
enough Return on Investment (ROI) for the spammer, but a sufficient enough ROI
for the person legitimately sending emails.
Training
Suppose whenever a link
comes through that *might* be a spam account, before you can look at the email,
you have to do some kind of training (this is along the lines of PhishMe, but it is not
corporate guided, so this is more "inform the masses"). Once you
finish the training, we could HIGHLIGHT links with big arrows that say
"THIS MIGHT BE SPAM OR SOMETHING NOT GOOD, WE DON'T RECOMMEND
CLICKING".
How would this help?
Prevention
Each of the email
providers has to scan each email as it arrives for spamminess. (Yes, I just
made this word up). Imagine if instead of having to parse 300 incoming emails,
it only has to parse one outgoing, deemed it as spam, and never sent it (of
course there would be ways to work around this, but basically it should involve
more work than spammers a spammers Return on Investment). Cut it off at the
source, less outgoing spam means less incoming spam, and it is a multiplier
when you stop it at the source.
Knowledge
People who don't
understand safe browsing behavior will either be prevented from even being able
to click on links from people they trust (but who weren't sent from people they
trust), or simply make it really obvious when you might be subjected to Spam.
By informing them repeatedly, it might just become second nature.
Hope you had a fun time
reading this.
I would love to hear
your feedback.
Monday, March 17, 2014
Convert Hidden text to Regular Text Word 2007
Came across a problem this morning converting Hidden text in word 2007 to normal text.
1. Enable the hidden text
a. windows ribbon, start menu, word options

b. Select Display, then Make sure Hidden Text is selected:

2. On the ribbon, in the editing box (typically far right), select the select drop down, then Select Text with Similar Formatting.

3. On the ribbon in the styles box click the arrow with the bar above it

and select clear formatting

There you go, now you know how to clear the formatting, there are some other ways floating around the internet explaining how to make hidden text normal text, but they didn't work for me. This worked.
Good luck and let me know if you find any other solutions that are easier (short cut keys would be awesome too).
1. Enable the hidden text

b. Select Display, then Make sure Hidden Text is selected:

2. On the ribbon, in the editing box (typically far right), select the select drop down, then Select Text with Similar Formatting.
3. On the ribbon in the styles box click the arrow with the bar above it

and select clear formatting

There you go, now you know how to clear the formatting, there are some other ways floating around the internet explaining how to make hidden text normal text, but they didn't work for me. This worked.
Good luck and let me know if you find any other solutions that are easier (short cut keys would be awesome too).
Subscribe to:
Posts (Atom)

