URL normalization

[stevenvachon]

These two URLs are most likely to be the same:

• http://host?var2&var1
• http://host?var1&var2

However, this library will cache both separately, which is a waste of resources.

There are many other examples of when URLs can be similar or exact, and situations where the server is not trusted and should be cached separately. I've been writing a library that accurately normalizes and provides "common" (trusted) and "careful" (untrusted) profiles.

Is this a feature you'd consider implementing?

[jpodwys]

Thank you for this post and for your patience! I agree that the two URLs you mentioned above should result in the same cache key. We may be able to solve this specific scenario without adding a dependency by making two changes:

1. Update the keygen function in utils.js to exclude all query params from the url property.
2. Use JSON.stringify's second argument called replacer to ensure the resulting string lists object keys in a deterministic order.

I'll be able to check whether this works soon.

[jpodwys]

For the second item in my previous comment, it looks like using this function as the replacer argument will achieve what we want in all browsers. So I believe this approach will ensure that unique query/header order will not result in unique cache keys.

[stevenvachon]

Query parameters are part of what constitutes a unique URL, though.

[jpodwys]

The keygen function creates an object containing the following properties

• request verb
• URL
• headers
• query params

The headers and query params are stored as objects then the whole object is stringified.

My proposal here is to remove the query params from the URL and then coerce the keys in the headers and query params objects into a deterministic order. So even though I mentioned removing query params from the URL, that data would still be present in generated key.