Blog Archives

Maps Web Application: Visual representation of Ida members

maps_front

About

For the Maps Web Application I was tasked with creating a system that could show the location and contact information of all the registered users from the Ida Institute’s website. Moreover, the system had to have a way to show the different groups that users belong to.

People within the countries had to be searchable by name.

There were several important decisions that had to be made to proceed with the development of this project:

  1. What kind of map visualization technology can be used?
  2. How to retrieve user data from the Typo3 CMS that is used for Ida’s website?
  3. How to modify the features in the Maps project
  4. How to show the groups that users belong to?
  5. How to show the members in each of the countries?
  6. How to embed the solution in the Ida’s website and control access to the data.

 

1. Map Technologies

Looking at the Map display technologies Google Maps usually immediately comes to mind. After examining their API it was decided not to use this solution because the location that was provided by the CMS was way too coarse (only the country was known).

I started looking at the alternatives and came by JVectorMap. This solution is JavaScript based and includes all the maps that the system requires.

2. Data Retrieval

Typo3 CMS provides a functionality to export the list of registered users in a CSV file.

First iteration of the system required manual uploading of the CSV file. This way of providing the updated user data worked but was really cumbersome on the person that had to keep the user’s list in the Maps system up-to-date.

I was not given access to the main database where user’s data is stored and there we no API to retrieve it by other simple means. This is where an idea to move the file retrieval to the Maps application was born.

I managed to login into the Typo3’s admin panel from the Maps code and retrieve the CSV file.

This was done by analyzing how the login process is handled in the CMS’s login page.

After filling in random Username and Password values and trying to login while capturing browser traffic using Fiddler the following WebForm values we determined to be required to perform a login into the CMS’s backend and get access to the people list.

maps_login

The challenge and the userident fields were changing when trying to login with the same credentials.

Closer inspection of the page source revealed that there is a form that has the required fields set up as hidden inputs.

maps_login_form

 

The challenge  value can be extracted by executing a GET request to the login page and parsing the returned text to find the challenge value. The only unclear bit was the way that userident field is used and where it’s value come from. Looking at the form definition there is a call to doChallengeResponse() javascript function before the form is submitted.

// THIS FUNCTION IS COPIED FROM THE TYPO3's LOGIN PAGE
function doChallengeResponse(superchallenged) { //
      password = document.loginform.p_field.value;
      if (password) {
            if (superchallenged) {
                  password = MD5(password); // this makes it superchallenged!!
            }
            str = document.loginform.username.value+":"+password+":"+document.loginform.challenge.value;
            document.loginform.userident.value = MD5(str);
            document.loginform.p_field.value = "";
            return true;
      }
}

It seems that the way the userident field is calculated consists of building a string that contains the username, a MD5 hash value of the password as well as the form’s challenge value separated by the colons (str = document.loginform.username.value+”:”+password+”:”+document.loginform.challenge.value)  and calculating a MD5 hash value of the aforementioned string. So I just re-implemented the userident calculation in C#.

// calculate UserIndent as per JS on the login page
var userIdent = HashMd5(userName + ":" + HashMd5(password) + ":" + challenge);

// ...

private string HashMd5(string input)
{
     return FormsAuthentication.HashPasswordForStoringInConfigFile(input, "MD5").ToLower();
}

After all the required values are ready to be submitted to the Typo3 a POST request is executed using System.Net.Http.HttpClient class and the authentication cookie is used for the next request that retrieves the the CSV file.

The whole process is logged in real time and can be viewed later by the system administrators (emails in the image below are obfuscated for privacy reasons).

maps_log

 

3. Settings Panel

To make the web application customizable a simple settings panel was created that allowed to manually upload the CSV file, perform country name mapping, select visible groups (explained later) as well as forcing the data refresh (usually it happens every few days).

maps_settings

 

4. Groups

maps_groups

The groups are created by parsing the CSV file and extracting all the unique groups that people belong to. All new groups are then stored in the Maps database as separate objects with their visibility set to hidden. This can be changed form the Settings Panel and it makes the group visible in the front page.

After website users click on the group map results are filtered to include only the Ida Members belonging to that particular group.

5. Ida Members

maps_members

Once a country is selected on the map website users are redirected to a page showing a list of all the members in that country. Since the height of the user boxes could not be predetermined because of different image size and different amount of data the layout had to adapt to different size boxes. Masonry library was used to take care of it. It arranges the grid elements into the most fitting positions.

6. Embedding the Solution

The Maps Web Application was included in Ida CMS’s page as an iframe. The only problem is that the height of the iframe is not know before the page loads. This issue was mitigated using IFrame Resizer libary. After the page is loaded it dynamically adjusts the frame height to match the height of the content inside the iframe.

Conclusion

At the time of writing this post, the project is more or less finished and waiting for the approval to be integrated into the main Ida’s site. The solution for automatically getting the users from the CMS is probably a bit hacky, but without access to data  or asking the CMS administrators to build an API for it (which introduces additional cost) this is probably the easiest way to do it.

Advertisements

Ida Newsletter with cms

newsletter_front

About

The initial version of the the mobile-friendly newsletter (link) required direct manipulation of HTML to create new instances. This was a bit cumbersome and was not user friendly.

The company asked me to develop a simple CMS system that would allow for easier management of the newsletters.

After looking at the existing CMS solutions (DNN, Umbraco, etc) they seemed to be an overkill for such task. I decided to develop the whole thing from scratch using ASP.NET MVC4 framework.

Design for the admin pages was not important, so it’s using more or less the default templates that come with the MVC 4 Internet project in Visual Studio 2012.

You can see the final result here: http://idainstitute.com/newsletter/.

Log in Screen

newsletter_loginTo limit access to the back-end system the default forms authentication scheme was used. After logging in the users are redirected to the Newsletters Overview screen.

Newsletters Overview Screen

newsletter_overview

 

In the Newsletters Overview screen it is possible to get access to all of the newsletters in the system, edit them  as well as create new newsletters.

All of the newsletters are sorted by the year and then by the quarter in the descending order.

 Newsletters Edit Screen

newsletter_edit

The edit screen shows the general information about the newsletter that can be modified (like title, year and quarter) as well as lists all the posts in the newsletter.

The posts can be deleted, edited and there’s a possibility to create new ones. The system currently does not support sorting of the posts after the fact they were added to the system.

Post Screens

newsletter_post_edit

Creating and Editing screens for the posts are more or less the same. The provide the functionality to modify the parts of the posts that are visible on the main page. The Content field is using TinyMCE WYSIWYG editor that allows for some customization of the post text.

Other issues

SmarterAsp.net is used to host the solution and they provide a really ugly URL. The company did not want to buy another domain (or use a sub-domain) for the newsletter so I developed a little hack to make it appear that the newsletter is coming form the main Ida’s domain (see the code below).

The PHP script loads the newsletter in an iframe thus masking the URL and making it appear as idainstitute.com/newsletter. 

<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
<title>Ida Institute | Newsletter</title>
</head>
<body style="margin: 0; background: #efeeef;">
< iframe
src='http://idainstitute-001-site2.smarterasp.net/?Year=<?php echo $_GET['Year']; ?>&Quarter=<?php echo $_GET['Quarter'];?>&forceMobile=<?php echo $_GET['forcemobile'];?>'
frameborder='0'
scrolling='no'
width='100%'

height='1000px'
id='newsletter'
>
</iframe>

<script src="http://code.jquery.com/jquery-1.11.0.min.js"></script>
<script src="http://idainstitute.com/fileadmin/user_upload/v_javascript/iframeResizer.min.js"></script>
<script type="text/javascript">
$('#newsletter').iFrameResize({
autoResize:true
});
</script>
</body>
</html>