Category Archives: Projects
Maps Web Application: Visual representation of Ida members
About
For the Maps Web Application I was tasked with creating a system that could show the location and contact information of all the registered users from the Ida Institute’s website. Moreover, the system had to have a way to show the different groups that users belong to.
People within the countries had to be searchable by name.
There were several important decisions that had to be made to proceed with the development of this project:
- What kind of map visualization technology can be used?
- How to retrieve user data from the Typo3 CMS that is used for Ida’s website?
- How to modify the features in the Maps project
- How to show the groups that users belong to?
- How to show the members in each of the countries?
- How to embed the solution in the Ida’s website and control access to the data.
1. Map Technologies
Looking at the Map display technologies Google Maps usually immediately comes to mind. After examining their API it was decided not to use this solution because the location that was provided by the CMS was way too coarse (only the country was known).
I started looking at the alternatives and came by JVectorMap. This solution is JavaScript based and includes all the maps that the system requires.
2. Data Retrieval
Typo3 CMS provides a functionality to export the list of registered users in a CSV file.
First iteration of the system required manual uploading of the CSV file. This way of providing the updated user data worked but was really cumbersome on the person that had to keep the user’s list in the Maps system up-to-date.
I was not given access to the main database where user’s data is stored and there we no API to retrieve it by other simple means. This is where an idea to move the file retrieval to the Maps application was born.
I managed to login into the Typo3’s admin panel from the Maps code and retrieve the CSV file.
This was done by analyzing how the login process is handled in the CMS’s login page.
After filling in random Username and Password values and trying to login while capturing browser traffic using Fiddler the following WebForm values we determined to be required to perform a login into the CMS’s backend and get access to the people list.
The challenge and the userident fields were changing when trying to login with the same credentials.
Closer inspection of the page source revealed that there is a form that has the required fields set up as hidden inputs.
The challenge value can be extracted by executing a GET request to the login page and parsing the returned text to find the challenge value. The only unclear bit was the way that userident field is used and where it’s value come from. Looking at the form definition there is a call to doChallengeResponse() javascript function before the form is submitted.
// THIS FUNCTION IS COPIED FROM THE TYPO3's LOGIN PAGE function doChallengeResponse(superchallenged) { // password = document.loginform.p_field.value; if (password) { if (superchallenged) { password = MD5(password); // this makes it superchallenged!! } str = document.loginform.username.value+":"+password+":"+document.loginform.challenge.value; document.loginform.userident.value = MD5(str); document.loginform.p_field.value = ""; return true; } }
It seems that the way the userident field is calculated consists of building a string that contains the username, a MD5 hash value of the password as well as the form’s challenge value separated by the colons (str = document.loginform.username.value+”:”+password+”:”+document.loginform.challenge.value) and calculating a MD5 hash value of the aforementioned string. So I just re-implemented the userident calculation in C#.
// calculate UserIndent as per JS on the login page var userIdent = HashMd5(userName + ":" + HashMd5(password) + ":" + challenge); // ... private string HashMd5(string input) { return FormsAuthentication.HashPasswordForStoringInConfigFile(input, "MD5").ToLower(); }
After all the required values are ready to be submitted to the Typo3 a POST request is executed using System.Net.Http.HttpClient class and the authentication cookie is used for the next request that retrieves the the CSV file.
The whole process is logged in real time and can be viewed later by the system administrators (emails in the image below are obfuscated for privacy reasons).
3. Settings Panel
To make the web application customizable a simple settings panel was created that allowed to manually upload the CSV file, perform country name mapping, select visible groups (explained later) as well as forcing the data refresh (usually it happens every few days).
4. Groups
The groups are created by parsing the CSV file and extracting all the unique groups that people belong to. All new groups are then stored in the Maps database as separate objects with their visibility set to hidden. This can be changed form the Settings Panel and it makes the group visible in the front page.
After website users click on the group map results are filtered to include only the Ida Members belonging to that particular group.
5. Ida Members
Once a country is selected on the map website users are redirected to a page showing a list of all the members in that country. Since the height of the user boxes could not be predetermined because of different image size and different amount of data the layout had to adapt to different size boxes. Masonry library was used to take care of it. It arranges the grid elements into the most fitting positions.
6. Embedding the Solution
The Maps Web Application was included in Ida CMS’s page as an iframe. The only problem is that the height of the iframe is not know before the page loads. This issue was mitigated using IFrame Resizer libary. After the page is loaded it dynamically adjusts the frame height to match the height of the content inside the iframe.
Conclusion
At the time of writing this post, the project is more or less finished and waiting for the approval to be integrated into the main Ida’s site. The solution for automatically getting the users from the CMS is probably a bit hacky, but without access to data or asking the CMS administrators to build an API for it (which introduces additional cost) this is probably the easiest way to do it.
Laptop power jack replacement
Intro
My girlfriend’s laptop decided to act up… When moving the cable around the charging was being interrupted. First thought – broken cable, but no… It’s worse – broken power jack on the motherboard.
Video
Text transcript
Opening the case
The first part of the dis-assembly is quite simple: remove the cover that protects the RAM, HDD and WiFi module.
Hard disk comes out quite easily as well.
So does the WiFi Module.
To remove the body of the laptop a lot of screws need to be undone. I just followed them from one side and removed them one by one.
Please note, there is a tiny screw in the HDD hole. Without removing it it’s impossible to open up the case. Other tricky place is below the battery. There are two screws there.
Now it’s time to remove the keyboard. One just needs to touch the tabs at the top while lifting the keyboard up gently and it pops out. A wide white ribbon cable needs to be removed before removing the keyboard completely.
And then all the screws can be accessed and removed. One of those screws hold the DVD drive. There are three sneaky screws hiding under the drive.
After using some percussive maintenance and removing a screw or two that I missed, the top cover finally gave in.
Removing the motherboard
To remove the motherboard first gently disconnect the display cable as well as the speakers. There are no tabs holding them, so they just pop out.
Remove the tape, lift up the tab and remove the cable connecting USB hub and the audio jacks. There are 5 screws holding the motherboard, but they are all marked with small white arrows.
Replacing the connector
Now the fun part. Here you can see the broken one as well as the new connector from ebay.
To desolder the old one I am using a desoldering pump that I got from eBay. It costs only 10 bucks but it works like a charm. Just let it heat up and suck away.
A few pins did not want to give up, so I just used some pliers and it gave up. After cleaning up with some solder wick the board is ready for the new connector. Some solder and it looks like new.
Putting it all back together
To put it back together I just followed the dis-assembly in reverse. Motherboard with its 5 screws, then the cables, the lid. After putting on the lid I carefully connected the power button ribbon cable as well as the mouse-pad one. It is important not to forget the three small screws before putting the DVD drive back in. And the one under the HDD. After putting the keyboard back in it’s place the ribbon cable needs to be connected before closing it up.
It all works now! Time for a coffee.
Ida Newsletter
During this project I had to migrate an already existing flash version of the newsletter to a HTML-based one.
I had the design of the page, but all the features had to be implemented manually.
One of the most interesting features was making the animated half-circles on the page. Since they were supposed to move sideways and the movement had to be interesting, I opted out for making a script (written in JavaScript) that randomizes the lifetime of each of the circles (direction, speed and position).
At the end, I think, it ended up looking really awesome.
You can check out one of the newsletters here.
Dilemma game
I have created an HTML version of the Dilemma game that was developed by the Ida Institute. You can check it out at http://idainstitute.com/games/dilemma/. More info about the game here (requires registration).
Epic Re:load v2 review
Re:load v2 is an adjustable constant-current load.
A really cool thing is that it is self powered and can go to 3.3v keeping it’s current limiting capabilities. I went for the 0-6 Amp version since the price difference is minuscule and it seemed more useful in the future. According to the documentation it can handle 20 W or more with the larger heat-sink. Who does not like having some extra leeway…
Please see the full review as well as build process in my YouTube video:
Project BAC v.1.1
This post is about a project that took ages to built and I manage to loose in an unfortunate circumstances…
I always wanted to build a clock. And this is what came up…
The Idea
Create a modular clock that has interchangeable displays.
The Execution
Main controller
The main module for the clock is an ATmega328 microcontroller with a DS1307 Real Time Clock. The LCD displays current date and temperature as well as shows the setup menu.
Interaction with the clock is done by one rotary encoder that has a built-in push button. The state diagram for the menu can be found below.
Display Controllers
Time display (described below) is controller by a series of 74HC595 shift registers. Since there is a need for two digits per segment (by segment I mean hours, minutes and seconds) there are 6 of them in total connected in series. To update the time display I have to shift out the current state of the hours, minutes and seconds every second. Since the update rate is relatively slow no stress is applied to the microcontroller and it can do other tasks like displaying temperature readings, etc.
Time display
Time is displayed on a self-built 7 segment displays. There are 3 LED’s for the horizontal axis and 4 for the vertical one. By testing various combinations I found that this setup was the most appealing to my eyes. LED’s in a segment are connected in parallel and routed to the input connector at the bottom of the board (color-coded cable in the photo). The only problem I had with the displays was that they were really bright even on the lowest brightness setting (that can adjusted through the menu). To mitigate this problem I added some plexiglass that I painted white to diffuse the light a bit. Note to myself for the future projects: do not use clear LED’s for such applications…
More photos
A short clip showing the testing of the time display
Conclusion
I had a lot of fun working on this project. It was a really nice learning experience and I am planning on rebuilding the project in the future with some improvements.
Audio switcher
I’ve been looking for a while now to find a way to switch between multiple audio sources and output them on my speaker system.
Finally, I decided that the simple way is the best way :), so there’s what I came up with.
This is a 3P4T switch with four 3.5mm stereo female audio jacks acting as input and a 3.5mm stereo female audio jack that outputs the selected channel to the speakers. The knob points to the channel that is selected. Audio sources in this photo is my main PC and a Media laptop used for watching videos. There’s still two unused inputs left for connecting my phone/tablet/etc.
Now some photos from the build process:
I’ve spend approx. 100 DKK (17.5USD) for the parts (box, switch, knob, audio cable) that I did not have in my spare parts bin, but also had lots of fun building it, so, to my mind, it was worth the price.
ST7735 1.8″ TFT Dispay: SPI vs General connection
Intro
I got a ST7735 based lcd from ebay and decided to play with it a bit.
Setup
- Arduino UNO
- 1.8″ TFT LCD from ebay
- Some jumper cables
Code
Display driver library: https://github.com/adafruit/Adafruit-ST7735-Library
And the required graphics library: https://github.com/adafruit/Adafruit-GFX-Library
I am using the demo-sketches from the AdafruitST7735\examples\:
- graphicstest
- graphicstest_highspeed
Comparison itself
In the video you can see the communication protocol speed comparison.
Conclusion
SPI is the way to go 🙂
Project LIBMS: Li-Ion Battery Monitoring System
LIBMS by Vaidas Sirtautas is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. Permissions beyond the scope of this license may be available at https://sadiav.wordpress.com
Video
or you can watch in HD on Youtube.
Problem
Let’s start with a problem…
In order to have my phone as well as my BlueTooth audio headset working I had to carry two different chargers to workplace. Moreover, I needed to find an unused socket to plug them in. And that’s really annoying…
Idea
I could see two possible solutions:
- Buy a charger that uses batteries and provides 5V to charge my devices
- Build one of my own
And, of course, I went with choice #2. I had some Lithium Ion batteries lying around. And this was the starting point of LIBMS.
LIBMS
It would be crazy to just power devices through some voltage regulator. LiIon cells do not like to be discharged below 3V and without some additional circuitry it was impossible to check whether they were OK to continue providing power to the devices. (Well, I could carry a multimeter and measure each of them from time to time, but that would be crazy :))
I found some IC’s that are capable of monitoring LiIon batteries, but where’s the fun in that…
I had some Atmega328 chips laying around. They have a 6 channel 10-bit ADC (there is actually only one ADC, but built-in multiplexer expands the inputs to 6). Awesome.
Batteries
I connected 3 of the batteries in series and then that block with another block of three batteries in parallel. V1..V6 show my tap points that are used to measure each cells voltage. E.g.
- V(G2) = V2 – V1
- V(G3) = V6 – V(G2)
- …
- V(G6) = V6 – V(G5)
Voltage regulators
At the beginning of this project I had some problems with voltage regulation. I powered the microcontroller (uC) directly from the main voltage regulator (LM338), but when a device was connected that requires a lot of current the voltage suddenly dropped and uC restarted. Not good…
I solved this problem by introducing secondary LM7805 voltage regulator that is used to power the uC and LM338 takes care of the USB ports. Using this structure provided me with a cool way for controlling the USB ports from uC.
Browsing through the LM338’s data-sheet I found an example where the regulator can be controller using a TTL signal. Perfect. Even though it does not shut down completely (there is still ~1.25V on the output), it does the job.
Sensing part
Because Atmega328 can measure up to 5V max, I had to scale down the voltages of the cells. To do that I used voltage dividers. There are 5 of them, connected to ports A0..A4. Resistors (with values specified) are R1..R10 in the schematic.
Schematic & PCB layout
*Click on the images for hi-res version.
There is a 10K pull-up resistor (not shown in schematic) connected from BC337 base to Vcc to ensure that Voltage regulator stays off while uC is starting up.
Code
This code was written using Arduino IDE (www.arduino.cc)
int analogValues [5]; int voltages[6]; unsigned char i; // Loop variable byte LEDs = 0; byte tempLEDs = 0; boolean turnOn = true; boolean updated = false; boolean firstrun = true; byte portVal; byte LEDstates[] = { 0xFF, 0xDF, 0x9F, 0x1F, 0x0F, 0x07, 0x03, 0x01, 0x00}; void setup() { // Set Transistor's pin to output pinMode(8, OUTPUT); digitalWrite(8, HIGH); // Set LED pins to outputs pinMode(5, OUTPUT); pinMode(6, OUTPUT); pinMode(7, OUTPUT); pinMode(9, OUTPUT); pinMode(10, OUTPUT); pinMode(11, OUTPUT); pinMode(12, OUTPUT); pinMode(13, OUTPUT); // Set analog inputs pinMode(A0, INPUT); pinMode(A1, INPUT); pinMode(A2, INPUT); pinMode(A3, INPUT); pinMode(A4, INPUT); // Disable Pull-up resistors on analog pins digitalWrite(A0, LOW); digitalWrite(A1, LOW); digitalWrite(A2, LOW); digitalWrite(A3, LOW); digitalWrite(A4, LOW); for (i = 0; i < 9; i++) { tempLEDs = LEDstates[i]; updateLEDs(); delay(500); } delay(1000); } void updateLEDs() { PORTD = (PORTD & 0x1F) | (tempLEDs & 0xE0); PORTB = (PORTB & 0xC1) | ((tempLEDs & 0x1F) << 1); } // Scale because of voltage divider by 2.624 void loop() { if (turnOn && updated) { digitalWrite(8, LOW); updated = false; } if (!turnOn && updated) { // Serial.println("turn OFF"); digitalWrite(8, HIGH); updated = false; } // Read status of all batteries for (i = 0; i < 5; i++) { analogValues[i] = analogRead(i); delay(10); } voltages[0] = analogValues[4]; voltages[1] = analogValues[3] - voltages[0]; voltages[2] = analogValues[2] - voltages[1] - voltages[0]; voltages[3] = analogValues[1]; voltages[4] = analogValues[0] - voltages[3]; voltages[5] = analogValues[2] - voltages[4] - voltages [3]; // Take care of low voltage per cell situations // Minimum threshold for a cell is 3.0V. // Minimum value is 3/(5/1024) = 614.4 // Scaled because of voltage divider by 2.624, // so 3V = 614/2.624 = 234 units boolean stopLoop = false; i = 0; while ( i < 6 && !stopLoop) { if (voltages[i] < 234) { stopLoop = true; turnOn = false; updated = true; digitalWrite(8, LOW); // Enter infinite loop while (1) { }; } i++; } //Serial.println(analogValues[2], DEC); // Take care of LED's if (analogValues[2] > 920) // 11.8V tempLEDs = LEDstates[0]; else if (analogValues[2] > 889) // 11.4V tempLEDs = LEDstates[1]; else if (analogValues[2] > 858) // 11.0V tempLEDs = LEDstates[2]; else if (analogValues[2] > 827) // 10.6V tempLEDs = LEDstates[3]; else if (analogValues[2] > 796) // 10.2V tempLEDs = LEDstates[4]; else if (analogValues[2] > 765) // 9.8V tempLEDs = LEDstates[5]; else if (analogValues[2] > 734) // 9.4V tempLEDs = LEDstates[6]; else tempLEDs = LEDstates[7]; // Something changed in total battery voltage if (LEDs != tempLEDs) { LEDs = tempLEDs; updateLEDs(); /* My old idea... // First three LED's tempLEDs = tempLEDs >> 5; // Right shift by 5 to get first three bits tempLEDs = tempLEDs << 5; portVal = PORTD; portVal = portVal << 3; // Right shift to get rid of last three bits; portVal = portVal >> 3; // Left shift to restore first 5 bits to their positions PORTD = portVal | tempLEDs; // Restore temp value that changed because of all the shifting tempLEDs = LEDs; // Last five LED's portVal = PORTB; tempLEDs = tempLEDs << 3; // get rid of first three bits tempLEDs = tempLEDs >> 2; // position temp byte correctly if (turnOn) tempLEDs = tempLEDs | 1; PORTB = tempLEDs; // Restore temp value that changed because of all the shifting tempLEDs = LEDs; */ } if (firstrun) { updated = true; turnOn = true; firstrun=false; } // Delay before next run delay(1000); }
Eagle files
Will be uploaded later