Dinesh Balaji
Posted on • Updated on
#html #web #forms #webdev
The other day I was trying to write a REST endpoint in Go, which uploads the contents of a form submitted in a browser to another REST endpoint, in other words,
Form in Browser ----> My GO Rest API ----> Another REST API
While doing that I ended up learning some fundamentals of how HTML forms work. So thought it might be a good thing to share what I learned and hence the post.. :)
The encoding type of a form is determined by the attribute enctype
. It can have three values,
application/x-www-form-urlencoded
- Represents a URL encoded form. This is the default value ifenctype
attribute is not set to anything.multipart/form-data
- Represents a Multipart form. This type of form is used when the user wants to upload filestext/plain
- A new form type introduced in HTML5, that as the name suggests, simply sends the data without any encoding
Now, let us look at each form type with an example to understand them better.
As the name suggests, the data that is submitted using this type of form is URL endcoded. Take the following form,
<form action="/urlencoded?firstname=sid&lastname=sloth" method="POST" enctype="application/x-www-form-urlencoded"> <input type="text" name="username" value="sidthesloth"/> <input type="text" name="password" value="slothsecret"/> <input type="submit" value="Submit" /></form>
Here, you can see that the form is submitted to the server using a POST request, this means that it has a body. But how is the body formatted? It is URL encoded. Basically, a long string of (name, value)
pairs are created. Each (name, value)
pair is separated from one another by a & (ampersand)
sign, and for each (name, value)
pair, the name
is separated from the value
by an = (equals)
sign, like say,
key1=value1&key2=value2
For the above form, it would be,username=sidthesloth&password=slothsecret
Also, notice that we have some query parameters passed in the action URL, /urlencoded?firstname=sid&lastname=sloth
.
Don't the URL encoded body and the query parameters passed in the action URL look awfully similar? It's because they are similar. They share the same format discussed above.
Try creating an HTML file with the above code and see how it's submitted in the dev tools. Here is a snap,
The things to notice here are the Content-Type
header which says application/x-www-form-urlencoded
, the query string and the form fields are transferred to the server in the format as discussed above.
Note: Don't get confused by the term Form Data in the screen shot. It's just how Google Chrome represents form fields.
All is fine, but there is a little more to the encoding process. Let's introduce some spaces in the submitted values, take the below form which is the same as the previous one but has the firstname
value changed from sid
to sid slayer
and username
value changed from sidthesloth
to sid the sloth
.
<form action="/urlencoded?firstname=sid slayer&lastname=sloth" method="POST" enctype="application/x-www-form-urlencoded"> <input type="text" name="username" value="sid the sloth"/> <input type="text" name="password" value="slothsecret"/> <input type="submit" value="Submit" /></form>
Now try to submit the form and see how the form fields are transferred in the dev tools. Here is a dev tools snap in Chrome.
Clearly, you can see that the spaces are replaced by either '%20' or '+'. This is done for both the query parameters and the form body.
Read this to understand when + and %20 can be used. This encompasses the URL encoding process.
Multipart forms are generally used in contexts where the user needs files to be uploaded to the server. However, we'll just focus on simple text field based forms, as is it enough to understand how they work.
To convert the above form into a multipart form all you have to do is to change the enctype
attribute of the form tag from application/x-www-form-urlencoded
to multipart/form-data
.
<form action="/multipart?firstname=sid slayer&lastname=sloth" method="POST" enctype="multipart/form-data"> <input type="text" name="username" value="sid the sloth"/> <input type="text" name="password" value="slothsecret"/> <input type="submit" value="Submit" /></form>
Let's go ahead and submit it and see how it appears in the dev tools.
There are the two things to notice here, the Content-Type
header and the payload of the form request. Let's go through them one by one.
Content-Type Header
The value of the Content-Type
header is obviously multipart/form-data
. But it also has another value, boundary
. The value for this in the example above is generated by the browser, but the user can very well define it as well, say for example, boundary=sidtheslothboundary
. We'll get to see how it's useful in the next section.
Request Body
The request payload contains the form fields themselves. Each (name, value)
pair is converted into a MIME message part of the following format,
--<<boundary_value>>
Content-Disposition: form-data; name="<<field_name>>"
<<field_value>>
The above format is repeated for each (name, value)
pair.
Finally, the entire payload is terminated by the boundary
value suffixed with a --
. So the entire request looks like,
--<<boundary_value>>
Content-Disposition: form-data; name="<<field_name>>"
<<field_value>>
--<<boundary_value>>
Content-Disposition: form-data; name="<<field_name>>"
<<field_value>>
--<<boundary_value>>--
Now, we see how the boundary value is used.
In the case of an application/x-www-form-urlencoded
form, the &
ampersand kind of acts as a delimiter between each (name, value)
pair, enabling the server to understand when and where a parameter value starts and ends.
username=sidthelsloth&password=slothsecret
In the case of a multipart/form-data
form, the boundary value serves this purpose. Say if the boundary value was XXX
, the request payload would look like,
--XXX
Content-Disposition: form-data; name="username"
sidthesloth
--XXX
Content-Disposition: form-data; name="password"
slothsecret
--XXX--
The hyphens themselves are not part of the boundary value but rather needed as part of the request format. The Content-Type
header for the above request would be,
Content-Type: multipart/form-data; boundary=XXX
This allows the browser to understand, when and where each field starts and ends.
These forms are pretty much the same as the URL encoded forms, except that the form fields are not URL encoded when sent to the server. These are not used widely in general, but they have been introduced as a part of the HTML 5 specification.
Avoid using them as they meant for human understanding and not for machines.
As quoted from the spec,
Payloads using the text/plain format are intended to be human readable. They are not reliably interpretable by computer, as the format is ambiguous (for example, there is no way to distinguish a literal newline in a value from the newline at
the end of the value).
Hope, I was clear in explaining what I learnt..See you in the next one guys..Peace.. :)
Get to know more about me on my website..✨
Top comments (23)
Subscribe
David Klempfner
David Klempfner
-
Joined
• Sep 11 '19
- Copy link
Great post, well explained!
Btw minor grammar mistake: "Represents an URL encoded form". It should be "a URL". "An" is only used before nouns that start phonetically with a vowel (URL is pronounced "yew-aar-el").
Dinesh Balaji
Dinesh Balaji
Web/Hybrid Mobile Application developer | Open Source Enthusiast | Cat and Ice Cream Lover
-
Email
www.sidthesloth@gmail.com
-
Location
Chennai, India
-
Education
Information Technology
-
Work
Senior Software Engineer at Ernst & Young GDS
-
Joined
• Sep 1 '20
- Copy link
Have made the edit, thanks.
Shubham Battoo
Shubham Battoo
Making the web with 100% recycled pixels.
-
Location
India
-
Work
Software Engineer
-
Joined
• Aug 12 '18
- Copy link
Good read.
Dinesh Balaji
Dinesh Balaji
Web/Hybrid Mobile Application developer | Open Source Enthusiast | Cat and Ice Cream Lover
-
Email
www.sidthesloth@gmail.com
-
Location
Chennai, India
-
Education
Information Technology
-
Work
Senior Software Engineer at Ernst & Young GDS
-
Joined
• Aug 12 '18
- Copy link
Thanks.. :)
Gabriel José
Gabriel José
TS/JS Backend Developer
-
Location
Brazil, Anápolis-GO
-
Work
Backend Developer at Wiser Educação
-
Joined
• Apr 8 '21
- Copy link
Great post! I've only one question about how can we get a file's content, without parsing the request body into a string. I've tried parsing it, got succeed but the result takes a too long time to complete.
fonzane
fonzane
-
Joined
• Jun 3 '19
- Copy link
How do I make the POST-Request and its details visible in the chrome dev-tools? Whenever I submit the form, I only see the requests to the files that belong to the "page not found" page in the network tab of chrome.
How did you do it?
davidchen0310
davidchen0310
-
Joined
• Dec 6 '18
- Copy link
It's comprehensive and I appreciate it, but I found some typos.
"Avoid using them as they meant for human understanding and for machines."
->
"Avoid using them as they meant for human understanding and not for machines."
z2lai
z2lai
I'm aspiring to make programming my bread and butter
-
Joined
• Mar 19 '19
- Copy link
->
"Avoid using them as they are meant for human understanding and not for machines."
Dinesh Balaji
Dinesh Balaji
Web/Hybrid Mobile Application developer | Open Source Enthusiast | Cat and Ice Cream Lover
-
Email
www.sidthesloth@gmail.com
-
Location
Chennai, India
-
Education
Information Technology
-
Work
Senior Software Engineer at Ernst & Young GDS
-
Joined
• Sep 1 '20
- Copy link
Have made the edit, thanks for pointing it out.
Diego Lopes
Diego Lopes
I'm just a person who wants to learn as much as I can of web development with amazing people that loves what they do.
-
Location
Brazil
-
Work
Front end developer at Resolv Risk
-
Joined
• Jan 4 '21
- Copy link
You helped me a lot!
Dinesh Balaji
Dinesh Balaji
Web/Hybrid Mobile Application developer | Open Source Enthusiast | Cat and Ice Cream Lover
-
Email
www.sidthesloth@gmail.com
-
Location
Chennai, India
-
Education
Information Technology
-
Work
Senior Software Engineer at Ernst & Young GDS
-
Joined
• Jan 5 '21
- Copy link
Happy to help.. :)
Ogo Okafor
Ogo Okafor
-
Joined
• Jun 15 '19
- Copy link
You just saved a life. Thanks Dinesh!
Nickolay Loshchinin
Nickolay Loshchinin
Working at GSPD (http://gspd.mobi). Growing the best team I have ever worked in.
-
Location
Saratov, Russia
-
Joined
• Oct 1 '19 • Edited on Oct 1 • Edited
- Copy link
Thanks for the article.
Avoid using them as they meant for human understanding and NOT for machines.
Is 'not' omitted?
Dinesh Balaji
Dinesh Balaji
Web/Hybrid Mobile Application developer | Open Source Enthusiast | Cat and Ice Cream Lover
-
Email
www.sidthesloth@gmail.com
-
Location
Chennai, India
-
Education
Information Technology
-
Work
Senior Software Engineer at Ernst & Young GDS
-
Joined
• Sep 1 '20
- Copy link
Yep, I have made the change. Thanks.
Jon
Jon
Hii guys, I'm a software engineerI'm very interested in technology since I was a kid :DCurrently, I'm working on the largest edutech company in Indonesia
-
Joined
• Mar 10 '19
- Copy link
great article :D..
Sung M. Kim
Sung M. Kim
Exploring the world of TypeScript, React & Node
-
Location
NYC - the Big 🍎
-
Education
SUNY Stony Brook
-
Joined
• Aug 12 '18
- Copy link
Thanks Dinesh for the post 🕺.
After using Form tags so many years, I've never paid attention to such small details.
Loved ❤️ how you laid out the article top to bottom.
Dinesh Balaji
Dinesh Balaji
Web/Hybrid Mobile Application developer | Open Source Enthusiast | Cat and Ice Cream Lover
-
Email
www.sidthesloth@gmail.com
-
Location
Chennai, India
-
Education
Information Technology
-
Work
Senior Software Engineer at Ernst & Young GDS
-
Joined
• Aug 13 '18
- Copy link
Thank you .. :)
Heshie Brody
Heshie Brody
Heshie Brody currently serves as a Technical Lead at Morgan and Morgan PA, where he oversees the development of both web and mobile client experiences.
-
Location
Brooklyn NY
-
Education
Flatiron School
-
Work
Lead Software Engineer at Morgan and Morgan PA
-
Joined
• Feb 28 '19
- Copy link
Great Resource!
Harshil Parmar
Harshil Parmar
Full Stack JS developer | Opensource | Freelance
-
Email
harshilparmar80@gmail.com
-
Location
Toronto,Canada
-
Work
Full stack Developer
-
Joined
• Feb 21 '21 • Edited on Feb 21 • Edited
- Copy link
Thanks @sidthesloth92
ChezCoder
ChezCoder
-
Joined
• Apr 28 '20
- Copy link
Is the "Content-Type" header mandatory for POST requests? What about for GET requests?
Ricky Chew
Ricky Chew
-
Joined
• May 21 '23
- Copy link
Great post, well explained!
Tracy Gilmore
Tracy Gilmore
After my first contact with a computer in the 1980's, I taught myself to program in BASIC and Z80 assembler. I went on to study Computer Science and have enjoyed a long career in Software Engineering.
-
Email
tracyggilmore@gmail.com
-
Location
Somerset, UK
-
Education
BSc (Hons) Computer Science
-
Work
Software Engineer specialising in web technologies, frontend and full stack (Node & xAMPP)
-
Joined
• May 21 '23
- Copy link
Hi Ricky and welcome to the Dev.to community.
Please take a moment to write a brief bio as it really helps people reading your comments to understand your perspective.
Kind regards Tracy
Naman Pathak
Naman Pathak
-
Joined
• Oct 6 '23
- Copy link
Good One,
Helped me a lot man
View full discussion (23 comments)
For further actions, you may consider blocking this person and/or reporting abuse