Category Archives: AJAX

Trigger Basic Auth Dialog with jQuery / AJAX

05 May 2017

We’re all familiar with basic authentication (or should be) – my experience with it has been via cPanel or doing it by hand on *nix servers. Once setup it just works. The other day I was integrating an app into a service where the dev environment had basic authentication on it. My AJAX calls were blocked (naturally) but did not trigger a username/password dialog to appear. I had assumed that the browser would just “do its thing”.

What is happening behind the scenes is that my AJAX call causes the server to return a 401 HTML error page. This is an unexpected response that jQuery can’t handle and that will cause the following error in the console:

$ Uncaught SyntaxError: Unexpected token < in JSON at position 0 at JSON.parse ()

jQuery will also return a status code “0” which, by the way, is a generic error from jQuery. As the response from the server was an HTML document (401 error page) jQuery couldn’t parse it (it was expecting the response to be JSON) thus the above error. Lacking any proper status code to use jQuery then just uses “0”.

Lastly if I inspect the response header I can see that WWW-Authenticate: Basic appears. That’s a clue as to what we should do to get things working which is to add a Authorization:Basic header to our AJAX request to begin with.

We can add the necessary header by specifying the beforeSend callback when we configure the AJAX request. Inside of the callback we can create the Authorization header that will be added to our request. See the following example:

  $.ajax(
    {
      method: 'POST',
      contentType: 'application/json',
      url: 'service URL here'
      data:'data here',
      beforeSend: function (xhr) {
        xhr.setRequestHeader ('Authorization', 'Basic xxxx');
      },
    })
    .done(function( data, textStatus, jqXHR ) {
      // do something here...
    })
    .fail(function( jqXHR, textStatus, errMsg ){
      // do something here...
    });

Here you can see that we are setting the “Authorization” header via xhr.setRequestHeader. The spec says that Basic should be followed by a base-64 representation of the user name and password to authenticate with but in this case we are using an arbitrary string. Since its obviously wrong the server will respond by asking for the correct credentials; the authentication dialog will appear.

As you can see there is some interaction going back and forth between the client and the server. If you’re attentive to the network tab in the Chrome console you can see the handshaking as it happens.

IE 11 Emulates older IE, But Not Really; Long-Running Processes in IE 7 & 8

05 Apr 2015

JavaScript engines in modern browsers are pretty fast – so fast that its easy to forget that not every browser is as capable as the latest Chrome or FireFox builds where JavaScript performance is concerned. In this case I experienced thread-locking that caused the “Script is taking too long – stop the script or wait” message in old IE but not in modern browsers. As I was using IE 11 to **emulate** older IE’s the issue didn’t appear until tested in genuine IE 7/8. While IE 11’s emulation of its older siblings gives you an idea of how well your layout changes it apparently uses its more modern JavaScriot engine.

My thread-locking problem came down to an XML file that was being served up to the page via AJAX which was then converted into JSON. While Chrome and Firefox didn’t take issue with the work and performed nicely while doing it it seemed that the process of converting the XML to JSON exceeded the built-in thresholds that genuine IE 7 & 8 had for a thread-blocking script. When the thresholds were hit those browsers warned users that the “page is unresponsive” when in fact it is not, its just not done crunching data yet.

The answer of course was to do the XML to JSON conversion server-side and then deliver the resulting JSON to the page.

Searching online I found a PHP script that worked pretty well for me and required a minor edit before I was able to mix it in to my work. Below is the minor edit that I made to the code found via the previous link:

//$xmlNode = simplexml_load_file('example.xml');  // old code located at the bottom of the PHP script
$xmlNode = simplexml_load_file($_GET["path"]); // new line that replaces the above

The edit makes the script look for a “path” variable from a GET request. For example, this GET tells the script where to find XML that needs to be returned as JSON:

your_server.com/xml2json.php?path=some_path/xml_file.xml

Here’s a full AJAX example using the above in jQuery:

$.ajax({
      url: 'your_server/xml2json.php?path=some_path/xml_file.xml',
      cache:false,
      dataType:'json'
   })
   .success(function(data,textStatus,jqXHR){
      // your success code here
   })
   .error(function(jqXHR,textStatus,err){
      // your error code here
   });

Since the dataType was configured to “json” jQuery takes care of turning the JSON into useable objects.

Its too easy to get spoiled by modern browsers – but we should always keep in mind that it is always better to do work on the server instead of relying on the browser to do it. This is just one example of many and a good reminder that we always have to think about older browsers as sure enough someone somewhere is going to use one.

On the topic of testing for old IE, if you’re a bit more tech savvy (just a tiny bit more) you can virtualize instances of different versions of Windows with different versions of IE.

Microsoft provides the VM’s for free at modern.ie. You can download Virtual Machine builds to run on popular platforms such as Oracle’s Virtual Box. As Microsoft provides the builds to run within Virtual box you don’t have to bother with installing the various flavors of Windows as its already been done for you.

Mixing jQuery and reCaptcha in Forms

06 Oct 2013

The business of collecting user data includes the ubiquitous form. The thing with forms is that they are the first to see abuse on the Internet and if left unprotected your inbox or database will soon be overrun with spam. One of many methods of protecting forms is the CAPTCHA. CAPTCHAs require that a user interpret a randomly-generated image by typing its value into a text field. Since automated programs (bots) can’t read the image they are stopped cold. Humans and their famously short attention spans happen to have enough bandwidth to tolerate a CAPTCHA or two. The Spammer variants, on the other hand, seem to lack the patience needed and will quickly move on to easier pickings of which there are untold thousands. So then, while a CAPTCHA is a minor inconvenience to your end-users it’s presence more than makes up for itself by its ability to completely stave off an avalanche of useless form posts.

Google provides a CAPTCHA service called reCAPTCHA which is the subject of this article.

The workflow I’ll be implementing is the following:

  1. User “submits” form
  2. The form is validated
  3. If form validation succeeds I then make an AJAX call to the reCaptcha service via a proxy to see if the user entered the correct reCaptcha value
  4. If an incorrect value was entered I’ll display an in-line message to the user
  5. If the correct value was entered I will submit the form

You can see this process in action below:

Register for a set of reCAPTCHA keys

You must first sign up to use reCAPTCHA by visiting http://www.google.com/recaptcha. After you register click the “Add Site” link and add your site. After you add a site you will get 2 keys – keep these handy, we’ll need them later.

Download the reCAPTCHA PHP Library

You will also need to download the reCAPTCHA PHP Library (I’m assuming that your site is on a PHP-compatible server).

Setting up a proxy

Since we plan on making AJAX calls to the reCAPTCHA service we know we will need a proxy as we would otherwise run afoul of the browser’s cross-domain policy. The way we will circumvent this is by creating a proxy. This proxy will handle communication between our web page and reCAPTCHA by forwarding our submitted form data to the reCAPTCHA service which in return responds to the proxy and then the proxy hands the response back to our web page.

The reCAPTCHA PHP library contains everything we need to setup our proxy. The library contains 5 files – we will be using 2 of them:

  • recaptchalib.php
  • example-captcha.php

The second file – example-captcha.php – will need to be modified for our purposes. Here is the original file (this is version 1.11) before any changes:


  
    
is_valid) { echo "You got it!"; } else { # set the error code so that we can display it $error = $resp->error; } } echo recaptcha_get_html($publickey, $error); ?>

To get to where we need to be we need to remove all html and make a couple of tiny changes to the PHP. I won’t walk through them in detail opting instead to just show you the finished edited file (the savvy among you will no doubt notice the differences) and to mention that the public and private reCAPTCHA keys are in use here:

is_valid) {
        echo "success";
    } else {
        echo "failure";
    }
}
?>

The line that echoes “success” is what we will be listening for in our AJAX call to determine if the reCAPTCHA verification was successful.

At this point your proxy is done. Go ahead and place your modified example-captcha.php and the unchanged recaptchalib.php on your server in the same folder as your form.

Add the jQuery and reCAPTCHA JS Libraries

You need to add these two lines within the head of your document (use whatever version of jQuery your project requires):

...


...

Setting up your form

The form code below is pretty basic – things to pay attention to:

  • This is a template for the reCAPTCHA functionality only – you still need to configure your form to work with your form remailer (i.e., gdform.php on GoDaddy, formmail.php, cgiemail or whatever the case might be)
  • Note the onsubmit=”return doForm()” in the FORM tag – this is the hook to our JavaScript that communicates to the reCAPTCHA server
  • There are 2 divs present:

    • <div id=”recap”></div> this is the wrapper for the reCaptcha itself
    • <div id=”err”>…</div> This is the in-line error message that we will display if the reCAPTCHA verification fails.
       
  • Note the style tag – we have a css class called “hidden” that is assigned to the reCAPTCHA error div. If there is an error we use jQuery to remove the class and show the error message to the user
     
...



Name:
...

Inserting the reCAPTCHA, validating Form and reCAPTCHA input

Next we need to insert the reCAPTCHA elements into the document. In the code that follows you can see that I’m inserting the reCAPTCHA on jQuery’s document.ready event.

When the form is submitted I first validate the form input which you can see in the doForm() function. If validation passes I then attempt to validate the reCAPTCHA via the recapVerify function.

Its all put together in the code snippet below (** add your public key where indicated in line 41 **):

...

...

And that should do it. Download all files here.

jQuery.parseJSON vs JSON.parse vs json_parse – Memory Profiles During AJAX Requests

10 Apr 2013

When doing an AJAX JSONP request with jQuery you rely on the jQuery.parseJSON method to parse the JSONP response into an object. While jQuery.parseJSON works most of the time I’ve seen it behave as if a memory leak is occurring. This isn’t new to jQuery.ajax() as it has somewhat of a history with memory leaks. The version I am using currently is jQuery 1.9.0, and with a particular project I am seeing a “stair-step” memory leak pattern where the usual ebb and flow of memory being allocated/deallocated appears as a predictable pattern. However, at a certain point there is a spike that itself never goes down which in effect sets the normal ebb/flow memory usage pattern at a higher level than previous. This happens 4 times within my web app, each spike setting a higher and higher average level of memory usage.

The Application

To understand what this is all about I should give a brief description of this particular application. What I have is an app that will be wrapped in PhoneGap. When this app launches on a mobile device it will download an XML document. That document contains data referencing all of the app’s downloadable assets. The app will then check to see if the data exists locally or not (if it did it would have been stored on the app via WebSQL) and if not (or there has been an update as determined by each asset’s timestamp) it will determine what files need to be downloaded and commence the download operation.

The “download” consists of AJAX calls to individual JSONP-type documents which have a structure similar to {date:value,name:value,data:value} where “data” is an image serialized to base64.

There are hundreds of these asset files. The individual files allow for the creation of a download/progress indicator where the file that is currently being downloaded is of course a percentage of the total.

The rest of this discussion deals with the downloading of the application’s assets as that is where the memory issue occurs.

jQuery.parseJSON

Anyone who uses jQuery.ajax() who works with JSON/JSONP will (or at least *should*) be using jQuery.parseJSON. The jQuery framework will take the JSON/JSONP server response and transparently transform it into a JavaScript Object to do with as the developer sees fit.

Initially I implemented the AJAX request exactly as I’ve done many times before not seeing the need to deviate from what has been a predictable pattern of interacting with JSONP server responses. Here is a code snippet of such a setup:

...
$.ajax({
   url:       server_path_here,
   type:      'GET',
   dataType:  'JSONP',
   timeout:    30000
...

Simple stuff, we’re requesting a JSONP response from the server and we have a timeout of 30 seconds.

Observe the following memory usage timeline obtained from Chrome via a script using the above AJAX configuration. The script “loops” in a synchronous fashion through an array of URL’s in an effort to cache the data within the browser. The workflow is: make request > download > parse > store via WebSQL > on txn success/failure move to next URL and start the process over:

jquery_memory_leak_complete_download

You can see quite plainly the pattern here and it is evident that we have a memory leak. Upon investigating the timeline I can see the exact AJAX calls where the leaks occur. The data contained within the JSONP payloads is the aforementioned base64 string and their lengths exceed 2 million characters. I now know the culprit but not why the leak happens but suspect that there is something wrong with jQuery’s handling of the data, in particular, jQuery.parseJSON.

JSON.parse

Some Googling on the topic reveals that there is some previous history on memory leaks while using jQuery to do AJAX calls. One note in particular mentioned the eval method being the root cause. jQuery’s JSON parser is based on work by Douglas Crockford who is well known within the JavaScript community. All web app developmers should be aware of him and his JSON parser script as it is commonly used/referenced by many frameworks. Anyway, I know from having visited D Crockford’s site many times that he offers more than one JSON parser and that one of them explicitly avoids using the eval method in its parsing routines.

Before I proceed to json_parse (the “eval-less” parser) I want to try JSON.parse. It follows that his JSON.parse script will also exhibit the memory leak when presented with JSON exceeding 2 million characters (and thus by extension any JavaScript library’s JSON parser based on his work, of which there are many). Here is the config for that test:

...
$.ajax({
   url:       server_path_here,
   type:      'GET',
   dataType:  'JSONP',
   timeout:    30000,
   converters: {'text json':JSON.parse} 
...

And the memory profile While using JSON.parse:

json.parse_memory_leak_complete_download

We can see the same thing happening – time to use Douglas Crockford’s alternative parser – json_parse.

json_parse

Douglas Crockford describes json_parse as “an alternative JSON parse function that uses recursive descent instead of eval.” Perfect – we want to get away from eval, here’s its implementation within jQuery:

...
$.ajax({
   url:       server_path_here,
   type:      'GET',
   dataType:  'JSONP',
   timeout:    30000,
   converters: {'text json':json_parse} 
...

And the resulting memory usage:

json_parse_memory_leak_non_optimized_complete_download

Thats what we’re looking for – you can see the spikes as the multi-million character JSON payloads get parsed and you can see GC happening afterwards. Memory gets reclaimed just the way we want it to.

From here we can delve further into optimizing the code responsible for these graphics – which is what I’ve done. A week or so of optimization and in some cases code-rewrite results in the following final download memory footprint for my web app:

json_parse_memory_leak_complete_download

Through streamlining a number of things – including removing all my attempts at debugging the issue – I’ve cut the download time and the overall memory foot print has been further improved – possible in large part to json_parse.

Conclusion

jQuery.parseJSON is useful for the vast majority of applications. However, if you are dealing with large JSON payloads – in excess of 1.8 to 2+ million characters in length you should use an alternative JSON parser such as json_parse. Do so and you will ensure that memory is reclaimed as needed and thus avoid app slugishness and crashes.

Here’s the link to get both JSON.parse and json_parse: https://github.com/douglascrockford/JSON-js.

Using the Google Maps API to Get Locations from Zip Codes

14 Sep 2012

Google offers many API’s, among them is the Maps API. In this example we’ll show all the code necessary to hit Google with a zip code to get the location in the form of City, State and Country.

As of June 22, 2016 you **must** apply for a Google Maps API key. All sites that use the service prior to June 22 will be grandfathered into the Google Maps service for keyless usage (Read this post from the Google Geo Developers Blog). Any new applications must include an API key so that Google can enforce rate limiting. If you see this error message: “MissingKeyMapError” – you are trying to use the Google Maps Service without a key – the fix is simply to acquire and start using an API key.

First thing to do is to reference Google’s Map API:

...

...

Next is a little form used to enter your zip code:

...
zip: Get Address
...

You can see that I have a link which fires a function called “getLocation()” – below you’ll find that function and the related code:

...


...

Thats it - open up the console in Chrome and notice the output as you enter different zip codes.