lwl | Less with links. |
| download: http://git.y1.nz/archives/lwl.tar.gz | |
| README | Files | Log | Refs |
commit f11c301b672f1e3d4f4f975638b09e2f9bcbb5fb parent b0165dd1904b64515b4a36f6db1242126e9b3260 Author: Emma Weaver <emma@y1.nz> Date: Mon, 6 Jul 2026 11:57:22 -0500 Wrote README, improved nN navigation, added commands, extracted controls to config.h Diffstat:
| M | Makefile | 6 | +++--- |
| M | README | 112 | +++++++++++++++++++++++++++++++++++++++++++++++++------------------------------ |
| M | TODO | 26 | +++++++++++++++----------- |
| M | config.h | 69 | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- |
| M | draw.c | 112 | +++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------- |
| M | main.c | 171 | ++++++++++++++++++++++++++++++++----------------------------------------------- |
| D | mode.h | 3 | --- |
| M | state.c | 368 | +++++++++++++++++++++++++++++++++++++++---------------------------------------- |
| M | state.h | 31 | +++++++++++-------------------- |
| M | test/8 | 2 | +- |
| A | test/current.html | 4539 | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | test/current.md | 3351 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | test/urls | 19 | +++++++++++++++++++ |
| M | text.c | 22 | +++++++++++++++------- |
14 files changed, 8418 insertions(+), 413 deletions(-)
diff --git a/Makefile b/Makefile @@ -15,11 +15,11 @@ clean: term.o: term.c term.h config.h $(CC) term.c -c -o term.o -state.o: state.c state.h config.h mode.h +state.o: state.c state.h config.h $(CC) state.c -c -o state.o -text.o: text.c config.h term.h text.h mode.h draw.h state.h +text.o: text.c config.h term.h text.h draw.h state.h $(CC) text.c -c -o text.o -draw.o: draw.c config.h text.h mode.h state.h +draw.o: draw.c config.h text.h state.h $(CC) draw.c -c -o draw.o diff --git a/README b/README @@ -1,46 +1,74 @@ LWL: less with links -Less, minus some of the Old Crust, plus link handling. +A tiny reimplementation of the Less pager, but with links. Controls: - By default, the controls are *very* similar to less. - See config.h to view/edit which keys do what. - -... -Controls Planning/Design: - Basic Navigation (DONE): - q - exit - jk - scroll up/down - hl - scroll right/left - gG - jump to top/bottom of file - /? - search forwards or backwards - del - delete char or exit search if no chars - enter - search - (highlight matches while active, instead of links) - nN - walk through search results - mM - cycle through search results (stays on screen) (TODO: change this?) - - Web Navigation: - nN - select next/previous link - mM - cycle through onscreen links - - links are generalized. They could be local files, web links, etc. - > so they must be handled generally. Perhaps via the config.h? - > how can you interact with a link? - - read it (always. in upper status bar?) - - run scripts with them. For example- - > >echo "$LINK" >> ~/bookmarks.txt - > >wget "$LINK" -qO - | digest | lwl - - scripts can either be hardcoded to a hotkey or written inline after ":" - *** How can I ensure links are handled safely? - (it's obviously very bad to just run "cat $LINK" lol....) - - (selected link is highlighted) - (*KEY FOR PREVIEWING FULL LINK?) - (if the selected link goes off screen...) - (when scrolling up, select earliest visible link) - (when scrolling down, select latest visible link) - tab - go to next visible link (+shift for backwards) - enter - follow the selected link (*HOW exactly?) - shift+enter - run script (*HOW exactly?) with the link - - - running arbitrary commands with them (Ex. cat $link >> my-bookmarks) + The controls are all descriptively defined in config.h. They are + intended to be edited by the user, if she so prefers. + + The following controls are included in the default configuration, + in addition to the most commonly used controls from less: + --------+-------------------------------------------------------------------- + nN | step through links, in addition to text search results. + --------+ + v | preview the full text of a long link. + --------+ + >! | can be used to run arbitrary commands using a link. + --------+ + enter | navigates to a link, using a separate program that handles + | browser-specific functionality such as search history, + | cookies, etc. + | + | By default, [bro](https://git.y1.nz/bro/files.html) is used. + --------+ + +Build: + Update config.mk and config.h to match your preferences, then run 'make' + to create the lwl binary. You can then install it by running 'make install' + with the necessary permissions. + +Dependencies: + This project only requires (TODO: C99-compatible) libc, and has no other + dependencies. + + As previously alluded to, the default controls rely on bro. This can be + trivially omitted if you don't want to use lwl as part of the bro web + browser (although that is the primary purpose of lwl). + + Note that lwl does not use any form of the curses library, despite being a + TUI. Because of this, it only supports standard vt100-like terminals. It + is however designed to be easy to port if you would like to run this using + some other kind of terminal (see term-controls.h). It uses only termios, + plus a minimal set of terminal control sequences: for moving the cursor, + clearing parts of the screen, and setting graphical rendition. + +-------------------------------------------------------------------------------- +Planning/Design: + q - exit + jk - scroll up/down + hl - scroll right/left + gG - jump to top/bottom of file + nN - walk through search results or links + + / - enter search mode + backspace - un-type one character + enter - do search + else - type character (if non-printing, ignores) + + > - enter fork mode + backspace - un-type one character + enter - set env LINK to current link, run command, then resume lwl + else - type character (if non-printing, ignores) + + ! - enter exec mode + backspace - un-type one character + enter - set env LINK to current link, run command, then terminate. + else - type character (if non-printing, ignores) + + v - preview full link (TODO: how else can I ensure the safety of links?) + +!!!browser-specific controls, [bro](git.y1.nz/bro) must be installed!!! + + enter - follow a link + backspace - navigate to previous page + diff --git a/TODO b/TODO @@ -1,11 +1,10 @@ - BUGS -[ ] optimize search (for example: "/e" in shakespeare lags Hard) - (*drawing with tons of highlights, is the bottleneck) -[ ] fix segfault on searching (switching between links/search results) -[ ] fix nested link ordering (right now "n" orders them in reverse) (*not sure if necessary!) -[ ] allocate required amounts of memory, for all buffers. - (instead of just const sizes...) +[ ] optimize drawing with tons of markups + (laggy examples: "/e" in shakespeare, test/6, test/7) + +[ ] fix nested link ordering + (right now "n" orders them in reverse) + (*not sure if necessary!) PLANNED FEATURES [x] Add make target "install" @@ -13,17 +12,22 @@ PLANNED FEATURES [x] Load file in 1d array instead of 2d screen-width-dependent one [x] Display markups/text-associated highlights [x] Add searching +[x] Extract control bindings into config.h [ ] Implement link-using commands - [ ] "e" to type arbitrary script with the link, + [x] '>' to type arbitrary script with the link, in the system editor, and then run it on quit. [x] "enter" to follow a link [ ] "backspace" to go back to the previous page (use bro?) - [ ] "TBD" to view raw webpage html -[ ] Extract control bindings into config.h -[ ] Extract terminal control sequences into a single header + [ ] 'v' to preview Very Long Links +[x] Update '>' to be able to run lwl/bro +[ ] Extract terminal control sequences into a single place [ ] Add usage: cat test/1 | lwl echo -e "a\nb\nc\n\ndef" | lwl (less achieves this by reading input from tty) (***NOT ACTUALLY sure if this is a good idea. Unneeded complexity...) [ ] Write man page + +THINGS THAT MIGHT BE NICE TO ADD +[ ] soft-wrap: It's kind of annoying to read hard-wrapped text... +[ ] padding on the sides (1 or 2 chars, would make reading nicer) diff --git a/config.h b/config.h @@ -1,12 +1,75 @@ +/* constant buffer lengths... */ +// TODO review these + see where malloc would make more sense. #define MAX_SCREEN_WIDTH 3000 #define MAX_SCREEN_HEIGHT 2000 - #define MAX_FILE_LENGTH 500000 /* in chars */ #define MAX_NEWLINES 50000 #define MAX_MARKUPS 50000 #define LINK_BUFFER_LENGTH 1000 - #define MAX_INPUT_LENGTH 300 -#define mode_indicators ":/" +/* appearance */ +#define LINK_COLOR COLOR_BLUE +#define SEARCH_COLOR COLOR_GREEN + +/* modes */ +#define MODE_GROUND 0 +#define MODE_SEARCH 1 +#define MODE_FORK 2 +#define MODE_EXEC 3 +#define mode_indicators ":/>!" + +/* non-printing characters */ +#define CR 0x0D /* enter */ +#define DEL 0x7F /* backspace */ +#define ETX 0x03 /* ^C */ + +/* key bindings */ +#define ACTIONS_GROUND \ + /* escapes */ \ + ACTION(EOF, ESCAPE) \ + ACTION('q', ESCAPE) \ + ACTION(ETX, ESCAPE) \ + \ + /* navigation */ \ + ACTION('g', SCROLL_TO(0)) \ + ACTION('G', SCROLL_TO(WRAP_END)) \ + ACTION('j', SCROLL_BY(1)) \ + ACTION('k', SCROLL_BY(-1)) \ + ACTION('d', SCROLL_BY(WRAP_H/2)) \ + ACTION('u', SCROLL_BY(-WRAP_H/2)) \ + ACTION(' ', SCROLL_BY(WRAP_H)) \ + ACTION('n', STEP_BY(1)) \ + ACTION('N', STEP_BY(-1)) \ + \ + /* change modes */ \ + ACTION('/', SET_MODE(MODE_SEARCH)) \ + ACTION('>', SET_MODE(MODE_FORK)) \ + ACTION('!', SET_MODE(MODE_EXEC)) \ + \ + /* browser-specific actions */ \ + ACTION(CR, EXEC_COMMAND("bro \"$LINK\"")) \ + ACTION('p', state->message = "Follow backlink") + +#define ACTIONS_SEARCH \ + ACTION(EOF, ESCAPE) \ + ACTION(ETX, ESCAPE) \ + ACTION(CR, DO_SEARCH) \ + ACTION(DEL, TYPE_BACKSPACE) \ + DEFAULT_ACTION(TYPE_CHAR) + +#define ACTIONS_FORK \ + ACTION(EOF, ESCAPE) \ + ACTION(ETX, ESCAPE) \ + ACTION(CR, FORK_COMMAND(state->input)) \ + ACTION(DEL, TYPE_BACKSPACE) \ + DEFAULT_ACTION(TYPE_CHAR) + +#define ACTIONS_EXEC \ + ACTION(EOF, ESCAPE) \ + ACTION(ETX, ESCAPE) \ + ACTION(CR, EXEC_COMMAND(state->input)) \ + ACTION(DEL, TYPE_BACKSPACE) \ + DEFAULT_ACTION(TYPE_CHAR) + diff --git a/draw.c b/draw.c @@ -2,30 +2,42 @@ #include "config.h" #include "text.h" -#include "mode.h" #include "state.h" void draw_status_line(State *state) { - if (state->message != NULL) - printf("\033[30;47m" "%s" "\033[0m", state->message); - else printf("%c", mode_indicators[state->mode]); - - if (state->mode == MODE_SEARCH) { - for (int i = 0; i < state->input_length; i++) - putchar(state->input[i]); + if (state->message != NULL) { + printf( + "\033[30;47m" "%s", + state->message + ); + } else if (state->mode != MODE_GROUND) { + printf( + "\033[1m" "%c%s" "\033[0m", + mode_indicators[state->mode], + state->input + ); + } else { + printf( + "\033[1m" "%c" "\033[0m", + mode_indicators[state->mode] + ); } - // TODO render command input + /* reset graphical rendition */ + printf("\033[0m"); } int color = -1; int active = 0; void -set_color(int new_color, int new_active) +set_color(Markup *markup) { + int new_color = markup == NULL ? COLOR_BLACK : markup->color; + int new_active = markup == NULL ? 0 : markup->active; + if (new_color == color && new_active == active) return; @@ -33,53 +45,63 @@ set_color(int new_color, int new_active) active = new_active; switch (color) { - case COLOR_RED: - if (active) fputs("\033[1;41;37m", stdout); + case COLOR_BLACK: + if (active) fputs("\033[1;47;30m", stdout); + else fputs("\033[0m", stdout); + break; + case COLOR_RED: + if (active) fputs("\033[0;41m", stdout); else fputs("\033[1;40;31m", stdout); break; - case COLOR_BLUE: + case COLOR_BLUE: if (active) fputs("\033[1;44;37m", stdout); else fputs("\033[1;40;34m", stdout); break; - case COLOR_GREEN: + case COLOR_GREEN: if (active) fputs("\033[1;42;37m", stdout); else fputs("\033[1;40;32m", stdout); break; - case COLOR_CYAN: - if (active) fputs("\033[1;46;37m", stdout); - else fputs("\033[1;40;36m", stdout); + case COLOR_YELLOW: + if (active) fputs("\033[1;43;30m", stdout); + else fputs("\033[1;40;33m", stdout); break; - case COLOR_MAGENTA: - if (active) fputs("\033[1;45;37m", stdout); + case COLOR_CYAN: + if (active) fputs("\033[1;46;30m", stdout); + else fputs("\033[40;36m", stdout); + break; + case COLOR_MAGENTA: + if (active) fputs("\033[1;45;30m", stdout); else fputs("\033[1;40;35m", stdout); break; - case COLOR_BLACK: - if (active) fputs("\033[47;30m", stdout); - else fputs("\033[0m", stdout); + case COLOR_WHITE: + if (active) fputs("\033[0;47;30m", stdout); + else fputs("\033[1;47;30m", stdout); break; } } -void -draw_markups(Markup **markups, int count, int index) +Markup * +get_markup_at(Markup **markups, int count, Markup *inner0, int index) { Markup *markup; - int inner_start = -1; - Markup *inner = NULL; + Markup *inner = inner0; + + // TODO iterate over all lists before printing the ""verdict"" for (int k = 0; k < count; k++) { markup = markups[k]; if (inner != NULL && inner->active && !markup->active) continue; if ( index >= markup->start && index < markup->start + markup->width && - (inner == NULL || markup->start > inner->start || markup->active) + ( + inner == NULL || + markup->start > inner->start || + markup->active + ) ) inner = markup; } - // TODO new color is - int inner_color = inner == NULL ? COLOR_BLACK : inner->color; - int inner_active = inner == NULL ? 0 : inner->active; - set_color(inner_color, inner_active); + return inner; } /* draws (wrapped) line index y in the cursor's current row. */ @@ -105,14 +127,30 @@ draw_line(FileContents *contents, Wrap *wrap, State *state, int y) return; } + Markup *inner; // TODO use write or fwrite instead maybe - for (int j = 0; j < line_length; j++) { // TODO no -3 + for (int j = 0; j < line_length; j++) { index = wrap->lines[y] + j; - // TODO do this a better way (PLEASE) - draw_markups(contents->highlights, contents->highlight_count, index); - draw_markups(contents->links, contents->link_count, index); - draw_markups(state->markups, state->markup_count, index); + inner = NULL; + inner = get_markup_at( + contents->highlights, + contents->highlight_count, + inner, + index + ); + inner = get_markup_at( + contents->links, + contents->link_count, + inner, + index); + inner = get_markup_at( + state->results, + state->result_count, + inner, + index + ); + set_color(inner); ch = line[j]; if (ch == '\n') break; @@ -120,6 +158,8 @@ draw_line(FileContents *contents, Wrap *wrap, State *state, int y) } printf("\033[0m"); /* reset graphic rendition */ + color = COLOR_BLACK; + active = 0; } void diff --git a/main.c b/main.c @@ -1,15 +1,31 @@ #include <stdio.h> /* printf, stdin/out/err, etc */ #include <termios.h> /* termios struct, tsetattr */ -#include <stdlib.h> /* exit, rand, srand */ -#include <unistd.h> /* execvp */ +#include <stdlib.h> /* exit, popen */ +#include <unistd.h> /* exec */ +#include <sys/wait.h> /* waitpid */ #include "config.h" #include "term.h" #include "text.h" -#include "mode.h" #include "draw.h" #include "state.h" +/* helper macros, for specifying keybindings in config.h */ +#define ESCAPE return 1 +#define SCROLL_TO(index) scroll_to(state, wrap, (index)) +#define SCROLL_BY(step) scroll_to(state, wrap, state->line_offset + (step)) +#define STEP_BY(step) next_markup(state, wrap, (step)) +#define SET_MODE(new_mode) set_mode(state, (new_mode)) +#define FORK_COMMAND(command) fork_command(state, (command)) +#define EXEC_COMMAND(command) exec_command(state, (command)) +#define TYPE_CHAR type_input_char(state, wrap, ch) +#define TYPE_BACKSPACE type_input_backspace(state) +#define DO_SEARCH do_search(contents, state, wrap) +#define WRAP_END wrap->line_count +#define WRAP_H wrap->height +#define ACTION(trigger, action) case trigger: action; break; +#define DEFAULT_ACTION(action) default: action; break; + /* GLOBAL VARS (sowwy,, ill cwean dis up l8r >w<) */ struct termios termios0; FileContents *contents; @@ -32,120 +48,76 @@ reset_termios(FILE *fd) tcsetattr(fileno(stdin), TCSANOW, &termios0); } -// TODO move this to a "debug" module and don't compile with it by default +// TODO catch SIGWINCH and trigger this void -change_width(int dw) +change_width(int nw, int nh) { int w = wrap->width, h = wrap->height; free(wrap); - wrap = wrap_to(contents, w + dw, h); + wrap = wrap_to(contents, nw, nh); } -/* just because this gets repeated 10,000,000 times.. */ -#define SCROLL_TO(index) scroll_to(state, wrap, (index)); - -int -handle_ground_input(char ch) +// TODO extract into proc (?) module, with all the forking and stuff +void +follow_link(State *state) { - switch (ch) { - case 'q': - case 0x03: /* ^C */ - return 1; - case 'g': - SCROLL_TO(0); - break; - case 'G': - SCROLL_TO(wrap->line_count+1 - wrap->height); - break; - case 'j': - SCROLL_TO(state->line_offset + 1); - break; - case 'k': - SCROLL_TO(state->line_offset - 1); - break; - case 'd': - SCROLL_TO(state->line_offset + wrap->height/2); - break; - case 'u': - SCROLL_TO(state->line_offset - wrap->height/2); - break; - case ' ': - SCROLL_TO(state->line_offset + wrap->height); - break; - case '/': - start_search(state); - break; - // TODO step for search results, cycle for links. - case 'n': cycle_markup_index(state, wrap, 1); break; - case 'N': cycle_markup_index(state, wrap, -1); break; - case '?': state->message = "Use / instead"; break; - case '<': state->message = "Use g instead"; break; - case '>': state->message = "Use G instead"; break; - - /* link controls */ - case 0x0D: /* enter */ - // follow_link(contents, state); - const char *link = state->markups[state->markup_index]->data; - reset_termios(stdin); - execlp("bro", "bro", link, NULL); - break; - case 0x7F: /* backspace */ - if (state->input_length == 0) { - state->message = "TODO: back to previous page D:"; - } - state->input_length = 0; - state->input[0] = '\0'; - break; + const char *link = state->markups[state->markup_index]->data; + if (link == NULL) { + state->message = "No link to follow"; + return; } + reset_termios(stdin); + execlp("bro", "bro", link, NULL); +} - // TODO make this debug-mode only.... - if (ch == ',') change_width(-1); - if (ch == '.') change_width(1); +// TODO move these to proc (?) module +void +exec_command(State *state, char *command) +{ + char *link = state->markups[state->markup_index]->data; + setenv("LINK", link, 1); + // TODO how can this be made safer... + + reset_termios(stdin); + printf("\n"); - return 0; + execlp("/bin/sh", "/bin/sh", "-c", command, NULL); } -int -handle_search_input(char ch) +void +fork_command(State *state, char *command) { - switch (ch) { - case 0x03: return 1; /* ^C */ - case 0x0D: /* enter */ - do_search(contents, state, wrap); - break; - case 0x7F: /* backspace */ - type_input_backspace(state); - break; - default: /* normal input */ - type_input_char(state, wrap, ch); - break; - } - return 0; + /* fork and execute in child */ + pid_t pid = fork(); + if (!pid) exec_command(state, command); + + /* parent: wait for child, then wait for keypress */ + waitpid(pid, NULL, 0); + printf("\033[0;1m" "[process terminated]" "\033[0m"); + set_termios_raw(stdin); + fflush(stdout); + fgetc(stdin); + state->mode = MODE_GROUND; } int handle_input(char ch) { - switch (state->mode) { - case MODE_GROUND: return handle_ground_input(ch); - case MODE_SEARCH: return handle_search_input(ch); + if (state->mode == MODE_GROUND) { + switch (ch) { ACTIONS_GROUND } + return 0; + } else if (state->mode == MODE_SEARCH) { + switch (ch) { ACTIONS_SEARCH } + return 0; + } else if (state->mode == MODE_FORK) { + switch (ch) { ACTIONS_FORK } + return 0; + } else if (state->mode == MODE_EXEC) { + switch (ch) { ACTIONS_EXEC } + return 0; } } -State * -init_state() -{ - State *ret = malloc(sizeof(State)); - ret->markup_count = 0; - ret->markup_index = 0; - ret->input = malloc(MAX_INPUT_LENGTH * sizeof(char)); - ret->input[0] = '\0'; - ret->input_length = 0; - ret->mode = MODE_GROUND; - ret->line_offset = 0; - return ret; -} - int main(int argc, const char *argv[]) { @@ -153,8 +125,6 @@ main(int argc, const char *argv[]) int result; int screen_width, screen_height; - srand((long int) &main); - state = init_state(); // TODO use getopt prolly @@ -176,10 +146,7 @@ main(int argc, const char *argv[]) wrap = wrap_to(contents, screen_width, screen_height); fclose(fin); - // TODO find a *good* way to do this, instead... - state->markups = contents->links; - state->markup_count = contents->link_count; - + set_markups(state, contents->links, contents->link_count); set_termios_raw(stdin); char ch; diff --git a/mode.h b/mode.h @@ -1,3 +0,0 @@ -#define MODE_GROUND 0 -#define MODE_SEARCH 1 -#define MODE_USE_LINK 2 diff --git a/state.c b/state.c @@ -1,112 +1,98 @@ -#include <string.h> +#include <string.h> /* strstr */ +#include <ctype.h> /* isprint */ +#include <stdlib.h> /* malloc */ #include "config.h" #include "state.h" -#include "mode.h" /* for printing dynamic messages into */ char message_buffer[MAX_SCREEN_WIDTH]; -#define START(markup_index) state->markups[markup_index]->start -#define CURRENT state->markup_index - -int -get_screen_start(State *state, Wrap *wrap) +State * +init_state() { - return wrap->lines[state->line_offset]; + State *ret = malloc(sizeof(State)); + ret->markup_count = 0; + ret->markup_index = 0; + ret->results = malloc(MAX_MARKUPS * sizeof(Markup *)); + ret->result_count = 0; + ret->input = malloc(MAX_INPUT_LENGTH * sizeof(char)); + ret->input[0] = '\0'; + ret->input_length = 0; + ret->mode = MODE_GROUND; + ret->line_offset = 0; + return ret; } -int -get_screen_end(State *state, Wrap *wrap) +/* functions for managing text input */ +void +set_mode(State *state, int mode) { - int last_line = state->line_offset + wrap->height; - if (last_line > wrap->line_count) last_line = wrap->line_count; - return wrap->lines[last_line]; + state->mode = mode; + state->input_length = 0; + state->input[0] = '\0'; } - -int -is_onscreen(State *state, Wrap *wrap, int index) +void +type_input_char(State *state, Wrap *wrap, char ch) { - int first = get_screen_start(state, wrap); - int last = get_screen_end(state, wrap); - return index >= first && index <= last; -} + if ( + state->input_length+1 >= wrap->width || + !isprint(ch) + ) + return; -int -get_first_markup(State *state, Wrap *wrap) -{ - int i = state->markup_count - 1; - int limit = get_screen_start(state, wrap); - for (;;) { - if (i-1 < 0 || START(i-1) < limit) - return i; - i--; - } + state->input[state->input_length++] = ch; + state->input[state->input_length] = '\0'; } - -int -get_last_markup(State *state, Wrap *wrap) +void +type_input_backspace(State *state) { - int i = 0; - int limit = get_screen_end(state, wrap); - for (;;) { - if ( - i+1 >= state->markup_count || - START(i+1) >= limit - ) return i; - i++; + if (state->input_length == 0) { + state->mode = MODE_GROUND; + return; } + state->input_length--; + state->input[state->input_length] = '\0'; } +/* then, some wrap-related utilities */ +// TODO extract wrap module from here (state.c) and text.c? int -is_selection_onscreen(State *state, Wrap *wrap) +get_screen_start(Wrap *wrap, int line_offset) { - return is_onscreen(state, wrap, START(CURRENT)); + return wrap->lines[line_offset]; } - int -get_furthest_onscreen_markup_index(State *state, Wrap *wrap, int delta_offset) +get_screen_end(Wrap *wrap, int line_offset) { - int scrolling_up = delta_offset < 0; - return scrolling_up ? - get_last_markup(state, wrap) : - get_first_markup(state, wrap); + int last_line = line_offset + wrap->height; + if (last_line > wrap->line_count) last_line = wrap->line_count; + return wrap->lines[last_line]; } - -void -set_markup_index(State *state, Wrap *wrap, int next) +int +is_onscreen(Wrap *wrap, int line_offset, int index) { - if (state->markup_count == 0) return; - - /* clamp the index */ - if (next < 0) next = 0; - if (next > state->markup_count - 1) - next = state->markup_count - 1; - - /* update colors */ - // TODO improve this... - state->markups[CURRENT]->active = 0; - state->markups[next]->active = 1; - - CURRENT = next; + int first = get_screen_start(wrap, line_offset); + int last = get_screen_end(wrap, line_offset); + return index >= first && index <= last; } - -/* the logic here is: */ -/* 2. else: -/* a. if scrolling up -> select last on screen */ -/* b. if scrolling down -> select first on screen */ -void -fix_markup_selection(State *state, Wrap *wrap, int delta_offset) +int +get_line_of(Wrap *wrap, int index) { - if (state->markup_count == 0) return; - if (is_selection_onscreen(state, wrap)) return; - - int furthest_index = get_furthest_onscreen_markup_index(state, wrap, delta_offset); - set_markup_index(state, wrap, furthest_index); + // TODO use binary instead of linear search + int line = 0; + for (;;) { + if ( + line+1 >= wrap->line_count || + wrap->lines[line+1] > index + ) break; + line++; + } + return line; } void -scroll_to_raw(State *state, Wrap *wrap, int offset) +scroll_to(State *state, Wrap *wrap, int offset) { /* clamp the offset */ if (offset + wrap->height - 1 > wrap->line_count) @@ -125,117 +111,66 @@ scroll_to_raw(State *state, Wrap *wrap, int offset) } } -/* scroll to, and update message + selected markup */ -void -scroll_to(State *state, Wrap *wrap, int offset) -{ - int delta_offset = offset - state->line_offset; - scroll_to_raw(state, wrap, offset); - - fix_markup_selection(state, wrap, delta_offset); -} - -/* update scroll to ensure that a specific index is onscreen. */ void -jump_to(State *state, Wrap *wrap, int index, int step) +unset_active(State *state) { - // TODO prevent scrolling when the index is already onscreen - // TODO put the target index at the most appropriate end of the screen - if (is_onscreen(state, wrap, index)) return; - - /* determine what line the target index is on */ - // TODO extract get_line_of(index)... - // TODO use binary instead of linear search (or cache? hm...) - int line = 0; - for (;;) { - if ( - line+1 >= wrap->line_count || - wrap->lines[line+1] > index - ) break; - line++; - } - - /* scroll so that "line" is in the middle of the screen */ - int scrolling_up = step < 0; - int pad = 2; - int target_line = scrolling_up ? - line - pad : - line - wrap->height + 1 + pad; - scroll_to_raw(state, wrap, target_line); -} - -void -step_markup_index(State *state, Wrap *wrap, int step) -{ - int next = CURRENT + step; - - if (state->markup_count == 0) return; - if ( - is_onscreen(state, wrap, START(CURRENT)) || ( - (START(CURRENT) < wrap->lines[state->line_offset]) != - (step < 0) - ) - ) set_markup_index(state, wrap, next); - - /* scroll to the start of the markup */ - jump_to(state, wrap, START(CURRENT), step); + state->markup_index >= state->markup_count || + state->markups == NULL + ) return; + state->markups[state->markup_index]->active = 0; } void -cycle_markup_index(State *state, Wrap *wrap, int step) +set_markup_index(State *state, int next) { - int next; - if (state->markup_count == 0) return; - /* if the next one is off-screen, loop back */ - next = CURRENT + step; - if ( - next >= state->markup_count || - next < 0 || - !is_onscreen(state, wrap, START(next)) - ) - next = get_furthest_onscreen_markup_index(state, wrap, step); - - if (!is_onscreen(state, wrap, START(next))) return; + // TODO do we actually still need to clamp this? + /* clamp the index */ + if (next < 0) next = 0; + if (next > state->markup_count - 1) + next = state->markup_count - 1; - set_markup_index(state, wrap, next); + /* update colors */ + unset_active(state); + state->markups[next]->active = 1; - snprintf( - message_buffer, wrap->width, - "Link: %s", state->markups[CURRENT]->data - ); - state->message = message_buffer; + state->markup_index = next; } void -start_search(State *state) +set_markups(State *state, Markup **new_markups, int new_count) { - state->mode = MODE_SEARCH; - state->input_length = 0; - state->input[0] = '\0'; + unset_active(state); + state->markups = new_markups; + state->markup_count = new_count; + set_markup_index(state, 0); } -void -type_input_char(State *state, Wrap *wrap, char ch) +int +get_first_markup(State *state, Wrap *wrap) { - if (state->input_length+1 >= wrap->width) - return; - - state->input[state->input_length++] = ch; - state->input[state->input_length] = '\0'; + int i = state->markup_count - 1; + int limit = get_screen_start(wrap, state->line_offset); + for (;;) { + if (i-1 < 0 || state->markups[i-1]->start < limit) + return i; + i--; + } } -void -type_input_backspace(State *state) +int +get_last_markup(State *state, Wrap *wrap) { - state->input[state->input_length] = '\0'; - state->input_length--; - if (state->input_length < 0) { - state->mode = MODE_GROUND; - state->input_length = 0; - state->input[0] = '\0'; + int i = 0; + int limit = get_screen_end(wrap, state->line_offset); + for (;;) { + if ( + i+1 >= state->markup_count || + state->markups[i+1]->start >= limit + ) return i; + i++; } } @@ -245,43 +180,106 @@ do_search(FileContents *contents, State *state, Wrap *wrap) char *start = contents->buffer; char *match; Markup *markup; - int matches; - /* reset search-related state */ + /* reset search state, free the old results */ state->mode = MODE_GROUND; - for (int i = 0; i < state->markup_count; i++) - free_markup(state->markups[i]); - state->markup_count = 0; - state->markup_index = 0; + set_markups(state, contents->links, contents->link_count); + for (int i = 0; i < state->result_count; i++) + free_markup(state->results[i]); + state->result_count = 0; /* return if the query is empty */ - if (state->input_length == 0) return; + if (state->input_length == 0) + return; - matches = 0; for (;;) { match = strstr(start, state->input); if (match == NULL) break; - matches++; markup = create_markup( match - contents->buffer, state->input_length, - COLOR_RED + SEARCH_COLOR ); - state->markups[state->markup_count++] = markup; + state->results[state->result_count++] = markup; start = match + state->input_length; } - if (matches == 0) { + if (state->result_count == 0) { state->message = "No matches found"; return; } - set_markup_index( - state, wrap, - get_first_markup(state, wrap) - ); + set_markups(state, state->results, state->result_count); state->message = message_buffer; - snprintf(message_buffer, wrap->width, "%d matches", state->markup_count); + snprintf(message_buffer, wrap->width, "%d matches", state->result_count); +} + +/* update scroll to ensure that a specific index is onscreen. */ +void +scroll_to_show(State *state, Wrap *wrap, int index) +{ + int new_line_offset, line_to_show = get_line_of(wrap, index); + int scrolling_up; + + /* if the line is already onscreen, do nothing */ + if ( + line_to_show >= state->line_offset && + line_to_show < state->line_offset + wrap->height + ) return; + + /* calculate the line offset, minimizing distance moved */ + scrolling_up = line_to_show < state->line_offset; + new_line_offset = scrolling_up ? + line_to_show: + line_to_show - wrap->height + 1; + scroll_to(state, wrap, new_line_offset); +} + +void +next_markup(State *state, Wrap *wrap, int step) +{ + int start; + int next = state->markup_index + step; + + if (state->markup_count == 0) return; + + start = state->markups[state->markup_index]->start; + if (!is_onscreen(wrap, state->line_offset, start)) { + next = step > 0 ? + get_first_markup(state, wrap) : + get_last_markup(state, wrap); + } + + if (next < 0) { + state->message = "Already at first"; + return; + } + if (next >= state->markup_count) { + state->message = "No more"; + return; + } + + set_markup_index(state, next); + + /* scroll to the start of the markup */ + start = state->markups[state->markup_index]->start; + scroll_to_show(state, wrap, start); + + if (state->markups[state->markup_index]->data != NULL) { + state->message = message_buffer; + int length = snprintf( + message_buffer, + wrap->width-3+1, /* -3 for "...", +1 for the '\0' */ + "Data: %s", + state->markups[state->markup_index]->data + ); + if (length >= wrap->width-2) + snprintf( + message_buffer + (wrap->width-3), + 3 + 1, /* 3 for "...", +1 for the '\0' */ + "..." + ); + } } diff --git a/state.h b/state.h @@ -16,32 +16,23 @@ typedef struct _s { char *input; int input_length; + Markup **results; + int result_count; + /* dynamic markups for showing text matches, etc */ Markup **markups; int markup_count; int markup_index; } State; -// TODO only define the externally useful things here. -// TODO simplify/polish link logic by separating -// TODO links, via list of onscreen links at any time, -// TODO search results, via global indices. -int get_first_screen_index(State *state, Wrap *wrap); -int get_last_screen_index(State *state, Wrap *wrap); -int is_onscreen(State *state, Wrap *wrap, int index); - -int get_first_onscreen_markup_index(State *state, Wrap *wrap); -int get_last_onscreen_markup_index(State *state, Wrap *wrap); -int get_furthest_onscreen_markup_index(State *state, Wrap *wrap, int delta_offset); - -void set_markup_index(State *state, Wrap *wrap, int new_markup_index); -void fix_markup_selection(State *state, Wrap *wrap, int delta_offset); -void scroll_to_raw(State *state, Wrap *wrap, int offset); -void scroll_to(State *state, Wrap *wrap, int offset); -void jump_to(State *state, Wrap *wrap, int index, int delta_index); -void step_markup_index(State *state, Wrap *wrap, int delta_index); -void cycle_markup_index(State *state, Wrap *wrap, int delta_index); -void start_search(State *state); +State *init_state(); + +void set_mode(State *state, int mode); void type_input_char(State *state, Wrap *wrap, char ch); void type_input_backspace(State *state); + +void scroll_to(State *state, Wrap *wrap, int index); + +void set_markups(State *state, Markup **markups, int markup_count); void do_search(FileContents *contents, State *state, Wrap *wrap); +void next_markup(State *state, Wrap *wrap, int step); diff --git a/test/8 b/test/8 @@ -3,7 +3,7 @@ parsing should work yet... Test test [this](second link) is more text (this isn't a link) but this next part [...](third link) is (was?) here's some stuff that's [not] links but kind of looks (a little) like links. -The links look like \[rababa](url). This 1 should get escaped! +The links look like \[rababa\](url). This 1 should get escaped! Here's another [paragraph](http://wiktionary.com/words/paragraph) that has a lot of links in it probably. it's here. It's pretty short. Whatever. diff --git a/test/current.html b/test/current.html @@ -0,0 +1,4539 @@ + +<!DOCTYPE html> + + + + + + + +<html data-bs-theme="auto" lang="en"> + <head> + + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <title> + + RFC 3986 - Uniform Resource Identifier (URI): Generic Syntax + + </title> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <link href="https://static.ietf.org/fonts/inter/import.css" rel="stylesheet"> + <link href="https://static.ietf.org/fonts/noto-sans-mono/import.css" rel="stylesheet"> + + <link rel="stylesheet" href="https://static.ietf.org/dt/12.68.0/ietf/css/document_html_referenced.css"> + + <script type="module" crossorigin="" src="https://static.ietf.org/dt/12.68.0/assets/embedded-027799ff.js"></script> +<link href="https://static.ietf.org/dt/12.68.0/assets/create-pinia-singleton-fd08aa1f.js" type="text/javascript" crossorigin="anonymous" rel="modulepreload" as="script" /> +<link href="https://static.ietf.org/dt/12.68.0/assets/Scrollbar-22ca35d5.js" type="text/javascript" crossorigin="anonymous" rel="modulepreload" as="script" /> + <script src="https://static.ietf.org/dt/12.68.0/ietf/js/document_html.js"></script> + <script src="https://static.ietf.org/dt/12.68.0/ietf/js/theme.js"></script> + + <link rel="alternate" type="application/atom+xml" title="Document changes" href="/feed/document-changes/rfc3986/"> + <meta name="description" + + content="Uniform Resource Identifier (URI): Generic Syntax (RFC 3986, )" + > + + +<link rel="apple-touch-icon" + sizes="180x180" + href="https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor-180.png"> +<link rel="icon" + sizes="32x32" + href="https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor-32.png"> +<link rel="icon" + sizes="16x16" + href="https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor-16.png"> +<link rel="manifest" href="/site.webmanifest"> +<link rel="mask-icon" + href="https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor-mask.svg" + color="#ffffff"> +<meta name="msapplication-TileColor" + content="#ffffff"> +<meta name="theme-color" + content="#ffffff"> + + + + + +<meta property="og:title" content="RFC 3986: Uniform Resource Identifier (URI): Generic Syntax"> +<meta property="og:url" content="https://datatracker.ietf.org/doc/html/rfc3986"> +<link rel="canonical" href="https://datatracker.ietf.org/doc/html/rfc3986"> +<meta property="og:site_name" content="IETF Datatracker"> +<meta property="og:description" content="A Uniform Resource Identifier (URI) is a compact sequence of characters that identifies an abstract or physical resource. This specification defines the generic URI syntax and a process for resolving URI references that might be in relative form, along with guidelines and security considerations for the use of URIs on the Internet. The URI syntax defines a grammar that is a superset of all valid URIs, allowing an implementation to parse the common components of a URI reference without knowing the scheme-specific requirements of every possible identifier. This specification does not define a generative grammar for URIs; that task is performed by the individual specifications of each URI scheme. [STANDARDS-TRACK]"> +<meta property="og:type" content="article"> + +<meta property="og:image" content="https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-card.png"> +<meta property="og:image:alt" content="Logo of the IETF"> +<meta property="article:section" content="IETF - Internet Engineering Task Force"> +<meta property="og:image:type" content="image/png"> +<meta property="og:image:width" content="1200"> +<meta property="og:image:height" content="630"> +<meta name="twitter:card" content="summary_large_image"> + +<meta property="article:author" content="Tim Berners-Lee"> +<meta property="article:author" content="Roy T. Fielding"> +<meta property="article:author" content="Larry M Masinter"> + + + + + <style> + + .diff-form .select2-selection__rendered { + direction: rtl; + text-align: left; + } + </style> + </head> + <body> + + <noscript><iframe class="status" title="Site status" src="/status/latest"></iframe></noscript> +<div class="vue-embed" data-component="Status"></div> + <div class="btn-toolbar sidebar-toolbar position-fixed top-0 end-0 m-2 m-lg-3 d-print-none"> + <div class="dropdown"> + <button class="btn btn-outline-secondary btn-sm me-1 dropdown-toggle d-flex align-items-center" + id="bd-theme" type="button" aria-expanded="false" data-bs-toggle="dropdown" + aria-label="Toggle theme"> + <i class="theme-icon-active bi bi-circle-half"></i> + </button> + + <ul class="dropdown-menu" aria-labelledby="bd-theme"> + <li> + <button type="button" class="dropdown-item d-flex align-items-center" + data-bs-theme-value="light" aria-pressed="false"> + <i class="me-2 opacity-50 theme-icon bi bi-sun-fill"></i> + Light<i class="bi bi-check2 ms-auto d-none"></i> + </button> + </li> + <li> + <button type="button" class="dropdown-item d-flex align-items-center" + data-bs-theme-value="dark" aria-pressed="false"> + <i class="me-2 opacity-50 theme-icon bi bi-moon-stars-fill"></i> + Dark<i class="bi bi-check2 ms-auto d-none"></i> + </button> + </li> + <li> + <button type="button" class="dropdown-item d-flex align-items-center active" + data-bs-theme-value="auto" aria-pressed="true"> + <i class="me-2 opacity-50 theme-icon bi bi-circle-half"></i> + Auto<i class="bi bi-check2 ms-auto d-none"></i> + </button> + </li> + </ul> + </div> + <button class="btn btn-outline-secondary btn-sm sidebar-toggle" + type="button" + data-bs-toggle="collapse" + data-bs-target="#sidebar" + aria-expanded="true" + aria-controls="sidebar" + aria-label="Toggle metadata sidebar" + title="Toggle metadata sidebar"> + <i class="bi bi-arrow-bar-left sidebar-shown"></i> + <i class="bi bi-arrow-bar-right sidebar-collapsed"></i> + </button> + </div> + <nav class="navbar bg-light-subtle px-1 fixed-top d-print-none d-md-none"> + <a class="nav-link ps-1" + href="/doc/rfc3986/"> + + RFC 3986 + + <br class="d-sm-none"> + + <span class="ms-sm-3 badge rounded-pill badge-std"> + + Internet Standard + + </span> + </a> + <button class="navbar-toggler p-1" + type="button" + data-bs-toggle="collapse" + data-bs-target="#docinfo-collapse" + aria-controls="docinfo-collapse" + aria-expanded="false" + aria-label="Show document information"> + <span class="navbar-toggler-icon small"></span> + </button> + <div class="navbar-nav navbar-nav-scroll overscroll-none collapse pt-1" id="docinfo-collapse"> + <div class="bg-light-subtle p-0"> + <table class="table table-sm table-borderless small"> + <tbody class="meta align-top"> + <tr> + <th scope="row"></th> + <th scope="row">Title</th> + <td class="edit"></td> + <td>Uniform Resource Identifier (URI): Generic Syntax</td> + </tr> + </tbody> + + + + + + + + +<tbody class="meta align-top "> + <tr> + <th scope="row">Document</th> + <th scope="row">Document type</th> + <td class="edit"></td> + <td> + + + + + + +<span class="text-success">RFC + + - Internet Standard + +</span> + + + + <br>January 2005 + + <br> + + <a class="btn btn-primary btn-sm my-1" + href="https://www.rfc-editor.org/errata_search.php?rfc=3986" title="Click to view errata." rel="nofollow"> + View errata + </a> + + + <a class="btn btn-sm btn-warning" + title="Click to report an error in the document." + href="https://www.rfc-editor.org/errata.php#reportnew" + target="_blank"> + Report errata + </a> + + + + <div>Updated by <a href="/doc/html/rfc8820" title="URI Design and Ownership">RFC 8820</a>, <a href="/doc/html/rfc7320" title="URI Design and Ownership">RFC 7320</a></div> + <div>Obsoletes <a href="/doc/html/rfc1808" title="Relative Uniform Resource Locators">RFC 1808</a>, <a href="/doc/html/rfc2732" title="Format for Literal IPv6 Addresses in URL's">RFC 2732</a>, <a href="/doc/html/rfc2396" title="Uniform Resource Identifiers (URI): Generic Syntax">RFC 2396</a></div> + <div>Updates <a href="/doc/html/rfc1738" title="Uniform Resource Locators (URL)">RFC 1738</a></div> + + + + <div> + Was + <a href="/doc/draft-fielding-uri-rfc2396bis/07/">draft-fielding-uri-rfc2396bis</a> + (individual in gen area) + </div> + + + + + + + + + + + + + + </td> + </tr> + + <tr> + <td></td> + <th scope="row">Select version</th> + <td class="edit"></td> + <td> + + + + + <ul class="revision-list pagination pagination-sm text-center flex-wrap my-0"> + + + + + <li class="page-item"> + <a class="page-link" + href="/doc/html/draft-fielding-uri-rfc2396bis-07" + rel="nofollow"> + 07 + </a> + </li> + + + + <li class="page-item rfc active"> + <a class="page-link" + href="/doc/html/rfc3986"> + RFC 3986 + </a> + </li> + + </ul> + + </td> + </tr> + + <tr> + <td></td> + <th scope="row">Compare versions</th> + <td class="edit"></td> + <td> + + + + +<form class="form-horizontal diff-form" + action="https://author-tools.ietf.org/iddiff" + method="get" + target="_blank"> + + <select class="form-select form-select-sm mb-1 select2-field" + data-max-entries="1" + data-width="resolve" + data-allow-clear="false" + data-minimum-input-length="0" + aria-label="From revision" + name="url1"> + + <option value="rfc3986"> + RFC 3986 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-07" selected> + draft-fielding-uri-rfc2396bis-07 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-06"> + draft-fielding-uri-rfc2396bis-06 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-05"> + draft-fielding-uri-rfc2396bis-05 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-04"> + draft-fielding-uri-rfc2396bis-04 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-03"> + draft-fielding-uri-rfc2396bis-03 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-02"> + draft-fielding-uri-rfc2396bis-02 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-01"> + draft-fielding-uri-rfc2396bis-01 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-00"> + draft-fielding-uri-rfc2396bis-00 + + </option> + + + </select> + + <select class="form-select form-select-sm mb-1 select2-field" + data-max-entries="1" + data-width="resolve" + data-allow-clear="false" + data-minimum-input-length="0" + aria-label="To revision" + name="url2"> + + <option value="rfc3986" selected> + RFC 3986 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-07"> + draft-fielding-uri-rfc2396bis-07 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-06"> + draft-fielding-uri-rfc2396bis-06 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-05"> + draft-fielding-uri-rfc2396bis-05 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-04"> + draft-fielding-uri-rfc2396bis-04 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-03"> + draft-fielding-uri-rfc2396bis-03 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-02"> + draft-fielding-uri-rfc2396bis-02 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-01"> + draft-fielding-uri-rfc2396bis-01 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-00"> + draft-fielding-uri-rfc2396bis-00 + + </option> + + + </select> + + <button type="submit" + class="btn btn-primary btn-sm" + value="--html" + name="difftype"> + Side-by-side + </button> + + <button type="submit" + class="btn btn-primary btn-sm" + value="--hwdiff" + name="difftype"> + Inline + </button> + +</form> + </td> + </tr> + + + <tr> + <td></td> + <th scope="row">Authors</th> + <td class="edit"> + + </td> + <td> + + + <span ><a + title="Datatracker profile of Tim Berners-Lee" + href="/person/timbl@w3.org" >T. Berners-Lee</a> <a + href="mailto:timbl%40w3.org" + aria-label="Compose email to timbl@w3.org" + title="Compose email to timbl@w3.org"> + <i class="bi bi-envelope"></i></a></span>, + + <span ><a + title="Datatracker profile of Roy T. Fielding" + href="/person/fielding@gbiv.com" >R. Fielding</a> <a + href="mailto:fielding%40gbiv.com" + aria-label="Compose email to fielding@gbiv.com" + title="Compose email to fielding@gbiv.com"> + <i class="bi bi-envelope"></i></a></span>, + + <span ><a + title="Datatracker profile of Larry M Masinter" + href="/person/lmm@acm.org" >L. Masinter</a> <a + href="mailto:lmm%40acm.org" + aria-label="Compose email to lmm@acm.org" + title="Compose email to lmm@acm.org"> + <i class="bi bi-envelope"></i></a></span> + + + <br> + <a class="btn btn-primary btn-sm mt-1" href="mailto:rfc3986@ietf.org?subject=rfc3986" title="Send email to the document authors">Email authors</a> + + </td> + </tr> + + + <tr> + <td></td> + <th scope="row"> + RFC stream + </th> + <td class="edit"> + + </td> + <td > + + + + + + + + +<img alt="IETF Logo" + class="d-lm-none w-25 mt-1" + + + + src="https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor-white.svg" + + + > + +<img alt="IETF Logo" + class="d-dm-none w-25 mt-1" + + + + src="https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor.svg" + + + > + + + + + </td> + </tr> + + <tr> + <td></td> + <th scope="row"> + Other formats + </th> + <td class="edit"> + </td> + <td> + + + <div class="buttonlist"> + + + <a class="btn btn-primary btn-sm" + + target="_blank" + href="https://www.rfc-editor.org/rfc/rfc3986.txt"> + + <i class="bi bi-file-text"></i> txt + + </a> + + + + <a class="btn btn-primary btn-sm" + + target="_blank" + href="https://www.rfc-editor.org/rfc/rfc3986.html"> + + <i class="bi bi-file-code"></i> html + + </a> + + + + + + <a class="btn btn-primary btn-sm" + + target="_blank" + href="https://www.rfc-editor.org/rfc/inline-errata/rfc3986.html"> + + <i class="bi bi-file-diff"></i> w/errata + + </a> + + + + <a class="btn btn-primary btn-sm" + + target="_blank" + href="/doc/rfc3986/bibtex/"> + + <i class="bi bi-file-ruled"></i> bibtex + + </a> + + +</div> + + + </td> + </tr> + + + + +</tbody> + <tr> + <th scope="row"></th> + <th scope="row"></th> + <td class="edit"></td> + <td> + <a class="btn btn-sm btn-warning mb-3" + target="_blank" + href="https://github.com/ietf-tools/datatracker/issues/new/choose"> + Report a bug + <i class="bi bi-bug"></i> + </a> + </td> + </tr> + </table> + </div> + </div> + </nav> + <div class="row g-0"> + <div class="col-md-9 d-flex justify-content-center lh-sm" + data-bs-spy="scroll" + data-bs-target="#toc-nav" + data-bs-smooth-scroll="true" + tabindex="0" + id="content"> + + <div class="rfcmarkup"> + <br class="noprint"> + <!-- [html-validate-disable-block attr-quotes, void-style, element-permitted-content, heading-level -- FIXME: rfcmarkup/rfc2html generates HTML with issues] --> + <div class="rfcmarkup"><pre>Network Working Group T. Berners-Lee +Request for Comments: 3986 W3C/MIT +STD: 66 R. Fielding +Updates: <a href="/doc/html/rfc1738">1738</a> Day Software +Obsoletes: <a href="/doc/html/rfc2732">2732</a>, <a href="/doc/html/rfc2396">2396</a>, <a href="/doc/html/rfc1808">1808</a> L. Masinter +Category: Standards Track Adobe Systems + January 2005 + + + <h1>Uniform Resource Identifier (URI): Generic Syntax</h1> + +Status of This Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (2005). + +Abstract + + A Uniform Resource Identifier (URI) is a compact sequence of + characters that identifies an abstract or physical resource. This + specification defines the generic URI syntax and a process for + resolving URI references that might be in relative form, along with + guidelines and security considerations for the use of URIs on the + Internet. The URI syntax defines a grammar that is a superset of all + valid URIs, allowing an implementation to parse the common components + of a URI reference without knowing the scheme-specific requirements + of every possible identifier. This specification does not define a + generative grammar for URIs; that task is performed by the individual + specifications of each URI scheme. + + + + + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 1]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-2"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +Table of Contents + + <a href="#section-1">1</a>. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-4">4</a> + <a href="#section-1.1">1.1</a>. Overview of URIs . . . . . . . . . . . . . . . . . . . . <a href="#page-4">4</a> + <a href="#section-1.1.1">1.1.1</a>. Generic Syntax . . . . . . . . . . . . . . . . . <a href="#page-6">6</a> + <a href="#section-1.1.2">1.1.2</a>. Examples . . . . . . . . . . . . . . . . . . . . <a href="#page-7">7</a> + <a href="#section-1.1.3">1.1.3</a>. URI, URL, and URN . . . . . . . . . . . . . . . <a href="#page-7">7</a> + <a href="#section-1.2">1.2</a>. Design Considerations . . . . . . . . . . . . . . . . . <a href="#page-8">8</a> + <a href="#section-1.2.1">1.2.1</a>. Transcription . . . . . . . . . . . . . . . . . <a href="#page-8">8</a> + <a href="#section-1.2.2">1.2.2</a>. Separating Identification from Interaction . . . <a href="#page-9">9</a> + <a href="#section-1.2.3">1.2.3</a>. Hierarchical Identifiers . . . . . . . . . . . . <a href="#page-10">10</a> + <a href="#section-1.3">1.3</a>. Syntax Notation . . . . . . . . . . . . . . . . . . . . <a href="#page-11">11</a> + <a href="#section-2">2</a>. Characters . . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-11">11</a> + <a href="#section-2.1">2.1</a>. Percent-Encoding . . . . . . . . . . . . . . . . . . . . <a href="#page-12">12</a> + <a href="#section-2.2">2.2</a>. Reserved Characters . . . . . . . . . . . . . . . . . . <a href="#page-12">12</a> + <a href="#section-2.3">2.3</a>. Unreserved Characters . . . . . . . . . . . . . . . . . <a href="#page-13">13</a> + <a href="#section-2.4">2.4</a>. When to Encode or Decode . . . . . . . . . . . . . . . . <a href="#page-14">14</a> + <a href="#section-2.5">2.5</a>. Identifying Data . . . . . . . . . . . . . . . . . . . . <a href="#page-14">14</a> + <a href="#section-3">3</a>. Syntax Components . . . . . . . . . . . . . . . . . . . . . . <a href="#page-16">16</a> + <a href="#section-3.1">3.1</a>. Scheme . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-17">17</a> + <a href="#section-3.2">3.2</a>. Authority . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-17">17</a> + <a href="#section-3.2.1">3.2.1</a>. User Information . . . . . . . . . . . . . . . . <a href="#page-18">18</a> + <a href="#section-3.2.2">3.2.2</a>. Host . . . . . . . . . . . . . . . . . . . . . . <a href="#page-18">18</a> + <a href="#section-3.2.3">3.2.3</a>. Port . . . . . . . . . . . . . . . . . . . . . . <a href="#page-22">22</a> + <a href="#section-3.3">3.3</a>. Path . . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-22">22</a> + <a href="#section-3.4">3.4</a>. Query . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-23">23</a> + <a href="#section-3.5">3.5</a>. Fragment . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-24">24</a> + <a href="#section-4">4</a>. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-25">25</a> + <a href="#section-4.1">4.1</a>. URI Reference . . . . . . . . . . . . . . . . . . . . . <a href="#page-25">25</a> + <a href="#section-4.2">4.2</a>. Relative Reference . . . . . . . . . . . . . . . . . . . <a href="#page-26">26</a> + <a href="#section-4.3">4.3</a>. Absolute URI . . . . . . . . . . . . . . . . . . . . . . <a href="#page-27">27</a> + <a href="#section-4.4">4.4</a>. Same-Document Reference . . . . . . . . . . . . . . . . <a href="#page-27">27</a> + <a href="#section-4.5">4.5</a>. Suffix Reference . . . . . . . . . . . . . . . . . . . . <a href="#page-27">27</a> + <a href="#section-5">5</a>. Reference Resolution . . . . . . . . . . . . . . . . . . . . . <a href="#page-28">28</a> + <a href="#section-5.1">5.1</a>. Establishing a Base URI . . . . . . . . . . . . . . . . <a href="#page-28">28</a> + <a href="#section-5.1.1">5.1.1</a>. Base URI Embedded in Content . . . . . . . . . . <a href="#page-29">29</a> + <a href="#section-5.1.2">5.1.2</a>. Base URI from the Encapsulating Entity . . . . . <a href="#page-29">29</a> + <a href="#section-5.1.3">5.1.3</a>. Base URI from the Retrieval URI . . . . . . . . <a href="#page-30">30</a> + <a href="#section-5.1.4">5.1.4</a>. Default Base URI . . . . . . . . . . . . . . . . <a href="#page-30">30</a> + <a href="#section-5.2">5.2</a>. Relative Resolution . . . . . . . . . . . . . . . . . . <a href="#page-30">30</a> + <a href="#section-5.2.1">5.2.1</a>. Pre-parse the Base URI . . . . . . . . . . . . . <a href="#page-31">31</a> + <a href="#section-5.2.2">5.2.2</a>. Transform References . . . . . . . . . . . . . . <a href="#page-31">31</a> + <a href="#section-5.2.3">5.2.3</a>. Merge Paths . . . . . . . . . . . . . . . . . . <a href="#page-32">32</a> + <a href="#section-5.2.4">5.2.4</a>. Remove Dot Segments . . . . . . . . . . . . . . <a href="#page-33">33</a> + <a href="#section-5.3">5.3</a>. Component Recomposition . . . . . . . . . . . . . . . . <a href="#page-35">35</a> + <a href="#section-5.4">5.4</a>. Reference Resolution Examples . . . . . . . . . . . . . <a href="#page-35">35</a> + <a href="#section-5.4.1">5.4.1</a>. Normal Examples . . . . . . . . . . . . . . . . <a href="#page-36">36</a> + <a href="#section-5.4.2">5.4.2</a>. Abnormal Examples . . . . . . . . . . . . . . . <a href="#page-36">36</a> + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 2]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-3"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + <a href="#section-6">6</a>. Normalization and Comparison . . . . . . . . . . . . . . . . . <a href="#page-38">38</a> + <a href="#section-6.1">6.1</a>. Equivalence . . . . . . . . . . . . . . . . . . . . . . <a href="#page-38">38</a> + <a href="#section-6.2">6.2</a>. Comparison Ladder . . . . . . . . . . . . . . . . . . . <a href="#page-39">39</a> + <a href="#section-6.2.1">6.2.1</a>. Simple String Comparison . . . . . . . . . . . . <a href="#page-39">39</a> + <a href="#section-6.2.2">6.2.2</a>. Syntax-Based Normalization . . . . . . . . . . . <a href="#page-40">40</a> + <a href="#section-6.2.3">6.2.3</a>. Scheme-Based Normalization . . . . . . . . . . . <a href="#page-41">41</a> + <a href="#section-6.2.4">6.2.4</a>. Protocol-Based Normalization . . . . . . . . . . <a href="#page-42">42</a> + <a href="#section-7">7</a>. Security Considerations . . . . . . . . . . . . . . . . . . . <a href="#page-43">43</a> + <a href="#section-7.1">7.1</a>. Reliability and Consistency . . . . . . . . . . . . . . <a href="#page-43">43</a> + <a href="#section-7.2">7.2</a>. Malicious Construction . . . . . . . . . . . . . . . . . <a href="#page-43">43</a> + <a href="#section-7.3">7.3</a>. Back-End Transcoding . . . . . . . . . . . . . . . . . . <a href="#page-44">44</a> + <a href="#section-7.4">7.4</a>. Rare IP Address Formats . . . . . . . . . . . . . . . . <a href="#page-45">45</a> + <a href="#section-7.5">7.5</a>. Sensitive Information . . . . . . . . . . . . . . . . . <a href="#page-45">45</a> + <a href="#section-7.6">7.6</a>. Semantic Attacks . . . . . . . . . . . . . . . . . . . . <a href="#page-45">45</a> + <a href="#section-8">8</a>. IANA Considerations . . . . . . . . . . . . . . . . . . . . . <a href="#page-46">46</a> + <a href="#section-9">9</a>. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-46">46</a> + <a href="#section-10">10</a>. References . . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-46">46</a> + <a href="#section-10.1">10.1</a>. Normative References . . . . . . . . . . . . . . . . . . <a href="#page-46">46</a> + <a href="#section-10.2">10.2</a>. Informative References . . . . . . . . . . . . . . . . . <a href="#page-47">47</a> + <a href="#appendix-A">A</a>. Collected ABNF for URI . . . . . . . . . . . . . . . . . . . . <a href="#page-49">49</a> + <a href="#appendix-B">B</a>. Parsing a URI Reference with a Regular Expression . . . . . . <a href="#page-50">50</a> + <a href="#appendix-C">C</a>. Delimiting a URI in Context . . . . . . . . . . . . . . . . . <a href="#page-51">51</a> + <a href="#appendix-D">D</a>. Changes from <a href="/doc/html/rfc2396">RFC 2396</a> . . . . . . . . . . . . . . . . . . . . <a href="#page-53">53</a> + <a href="#appendix-D.1">D.1</a>. Additions . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-53">53</a> + <a href="#appendix-D.2">D.2</a>. Modifications . . . . . . . . . . . . . . . . . . . . . <a href="#page-53">53</a> + Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-56">56</a> + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-60">60</a> + Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . <a href="#page-61">61</a> + + + + + + + + + + + + + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 3]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-4"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h2><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</h2> + + A Uniform Resource Identifier (URI) provides a simple and extensible + means for identifying a resource. This specification of URI syntax + and semantics is derived from concepts introduced by the World Wide + Web global information initiative, whose use of these identifiers + dates from 1990 and is described in "Universal Resource Identifiers + in WWW" [<a href="/doc/html/rfc1630">RFC1630</a>]. The syntax is designed to meet the + recommendations laid out in "Functional Recommendations for Internet + Resource Locators" [<a href="/doc/html/rfc1736">RFC1736</a>] and "Functional Requirements for Uniform + Resource Names" [<a href="/doc/html/rfc1737">RFC1737</a>]. + + This document obsoletes [<a href="/doc/html/rfc2396">RFC2396</a>], which merged "Uniform Resource + Locators" [<a href="/doc/html/rfc1738">RFC1738</a>] and "Relative Uniform Resource Locators" + [<a href="/doc/html/rfc1808">RFC1808</a>] in order to define a single, generic syntax for all URIs. + It obsoletes [<a href="/doc/html/rfc2732">RFC2732</a>], which introduced syntax for an IPv6 address. + It excludes portions of <a href="/doc/html/rfc1738">RFC 1738</a> that defined the specific syntax of + individual URI schemes; those portions will be updated as separate + documents. The process for registration of new URI schemes is + defined separately by [<a href="#ref-BCP35">BCP35</a>]. Advice for designers of new URI + schemes can be found in [<a href="/doc/html/rfc2718">RFC2718</a>]. All significant changes from <a href="/doc/html/rfc2396">RFC</a> + <a href="/doc/html/rfc2396">2396</a> are noted in <a href="#appendix-D">Appendix D</a>. + + This specification uses the terms "character" and "coded character + set" in accordance with the definitions provided in [<a href="#ref-BCP19">BCP19</a>], and + "character encoding" in place of what [<a href="#ref-BCP19">BCP19</a>] refers to as a + "charset". + +<h3><a class="selflink" id="section-1.1" href="#section-1.1">1.1</a>. Overview of URIs</h3> + + URIs are characterized as follows: + + Uniform + + Uniformity provides several benefits. It allows different types + of resource identifiers to be used in the same context, even when + the mechanisms used to access those resources may differ. It + allows uniform semantic interpretation of common syntactic + conventions across different types of resource identifiers. It + allows introduction of new types of resource identifiers without + interfering with the way that existing identifiers are used. It + allows the identifiers to be reused in many different contexts, + thus permitting new applications or protocols to leverage a pre- + existing, large, and widely used set of resource identifiers. + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 4]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-5"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + Resource + + This specification does not limit the scope of what might be a + resource; rather, the term "resource" is used in a general sense + for whatever might be identified by a URI. Familiar examples + include an electronic document, an image, a source of information + with a consistent purpose (e.g., "today's weather report for Los + Angeles"), a service (e.g., an HTTP-to-SMS gateway), and a + collection of other resources. A resource is not necessarily + accessible via the Internet; e.g., human beings, corporations, and + bound books in a library can also be resources. Likewise, + abstract concepts can be resources, such as the operators and + operands of a mathematical equation, the types of a relationship + (e.g., "parent" or "employee"), or numeric values (e.g., zero, + one, and infinity). + + Identifier + + An identifier embodies the information required to distinguish + what is being identified from all other things within its scope of + identification. Our use of the terms "identify" and "identifying" + refer to this purpose of distinguishing one resource from all + other resources, regardless of how that purpose is accomplished + (e.g., by name, address, or context). These terms should not be + mistaken as an assumption that an identifier defines or embodies + the identity of what is referenced, though that may be the case + for some identifiers. Nor should it be assumed that a system + using URIs will access the resource identified: in many cases, + URIs are used to denote resources without any intention that they + be accessed. Likewise, the "one" resource identified might not be + singular in nature (e.g., a resource might be a named set or a + mapping that varies over time). + + A URI is an identifier consisting of a sequence of characters + matching the syntax rule named <URI> in <a href="#section-3">Section 3</a>. It enables + uniform identification of resources via a separately defined + extensible set of naming schemes (<a href="#section-3.1">Section 3.1</a>). How that + identification is accomplished, assigned, or enabled is delegated to + each scheme specification. + + This specification does not place any limits on the nature of a + resource, the reasons why an application might seek to refer to a + resource, or the kinds of systems that might use URIs for the sake of + identifying resources. This specification does not require that a + URI persists in identifying the same resource over time, though that + is a common goal of all URI schemes. Nevertheless, nothing in this + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 5]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-6"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + specification prevents an application from limiting itself to + particular types of resources, or to a subset of URIs that maintains + characteristics desired by that application. + + URIs have a global scope and are interpreted consistently regardless + of context, though the result of that interpretation may be in + relation to the end-user's context. For example, "http://localhost/" + has the same interpretation for every user of that reference, even + though the network interface corresponding to "localhost" may be + different for each end-user: interpretation is independent of access. + However, an action made on the basis of that reference will take + place in relation to the end-user's context, which implies that an + action intended to refer to a globally unique thing must use a URI + that distinguishes that resource from all other things. URIs that + identify in relation to the end-user's local context should only be + used when the context itself is a defining aspect of the resource, + such as when an on-line help manual refers to a file on the end- + user's file system (e.g., "file:///etc/hosts"). + +<h4><a class="selflink" id="section-1.1.1" href="#section-1.1.1">1.1.1</a>. Generic Syntax</h4> + + Each URI begins with a scheme name, as defined in <a href="#section-3.1">Section 3.1</a>, that + refers to a specification for assigning identifiers within that + scheme. As such, the URI syntax is a federated and extensible naming + system wherein each scheme's specification may further restrict the + syntax and semantics of identifiers using that scheme. + + This specification defines those elements of the URI syntax that are + required of all URI schemes or are common to many URI schemes. It + thus defines the syntax and semantics needed to implement a scheme- + independent parsing mechanism for URI references, by which the + scheme-dependent handling of a URI can be postponed until the + scheme-dependent semantics are needed. Likewise, protocols and data + formats that make use of URI references can refer to this + specification as a definition for the range of syntax allowed for all + URIs, including those schemes that have yet to be defined. This + decouples the evolution of identification schemes from the evolution + of protocols, data formats, and implementations that make use of + URIs. + + A parser of the generic URI syntax can parse any URI reference into + its major components. Once the scheme is determined, further + scheme-specific parsing can be performed on the components. In other + words, the URI generic syntax is a superset of the syntax of all URI + schemes. + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 6]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-7"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h4><a class="selflink" id="section-1.1.2" href="#section-1.1.2">1.1.2</a>. Examples</h4> + + The following example URIs illustrate several URI schemes and + variations in their common syntax components: + + <a href="ftp://ftp.is.co.za/rfc/rfc1808.txt">ftp://ftp.is.co.za/rfc/rfc1808.txt</a> + + <a href="http://www.ietf.org/rfc/rfc2396.txt">http://www.ietf.org/rfc/rfc2396.txt</a> + + ldap://[2001:db8::7]/c=GB?objectClass?one + + mailto:John.Doe@example.com + + news:comp.infosystems.www.servers.unix + + tel:+1-816-555-1212 + + telnet://192.0.2.16:80/ + + urn:oasis:names:specification:docbook:dtd:xml:4.1.2 + + +<h4><a class="selflink" id="section-1.1.3" href="#section-1.1.3">1.1.3</a>. URI, URL, and URN</h4> + + A URI can be further classified as a locator, a name, or both. The + term "Uniform Resource Locator" (URL) refers to the subset of URIs + that, in addition to identifying a resource, provide a means of + locating the resource by describing its primary access mechanism + (e.g., its network "location"). The term "Uniform Resource Name" + (URN) has been used historically to refer to both URIs under the + "urn" scheme [<a href="/doc/html/rfc2141">RFC2141</a>], which are required to remain globally unique + and persistent even when the resource ceases to exist or becomes + unavailable, and to any other URI with the properties of a name. + + An individual scheme does not have to be classified as being just one + of "name" or "locator". Instances of URIs from any given scheme may + have the characteristics of names or locators or both, often + depending on the persistence and care in the assignment of + identifiers by the naming authority, rather than on any quality of + the scheme. Future specifications and related documentation should + use the general term "URI" rather than the more restrictive terms + "URL" and "URN" [<a href="/doc/html/rfc3305">RFC3305</a>]. + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 7]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-8"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h3><a class="selflink" id="section-1.2" href="#section-1.2">1.2</a>. Design Considerations</h3> + +<h4><a class="selflink" id="section-1.2.1" href="#section-1.2.1">1.2.1</a>. Transcription</h4> + + The URI syntax has been designed with global transcription as one of + its main considerations. A URI is a sequence of characters from a + very limited set: the letters of the basic Latin alphabet, digits, + and a few special characters. A URI may be represented in a variety + of ways; e.g., ink on paper, pixels on a screen, or a sequence of + character encoding octets. The interpretation of a URI depends only + on the characters used and not on how those characters are + represented in a network protocol. + + The goal of transcription can be described by a simple scenario. + Imagine two colleagues, Sam and Kim, sitting in a pub at an + international conference and exchanging research ideas. Sam asks Kim + for a location to get more information, so Kim writes the URI for the + research site on a napkin. Upon returning home, Sam takes out the + napkin and types the URI into a computer, which then retrieves the + information to which Kim referred. + + There are several design considerations revealed by the scenario: + + o A URI is a sequence of characters that is not always represented + as a sequence of octets. + + o A URI might be transcribed from a non-network source and thus + should consist of characters that are most likely able to be + entered into a computer, within the constraints imposed by + keyboards (and related input devices) across languages and + locales. + + o A URI often has to be remembered by people, and it is easier for + people to remember a URI when it consists of meaningful or + familiar components. + + These design considerations are not always in alignment. For + example, it is often the case that the most meaningful name for a URI + component would require characters that cannot be typed into some + systems. The ability to transcribe a resource identifier from one + medium to another has been considered more important than having a + URI consist of the most meaningful of components. + + In local or regional contexts and with improving technology, users + might benefit from being able to use a wider range of characters; + such use is not defined by this specification. Percent-encoded + octets (<a href="#section-2.1">Section 2.1</a>) may be used within a URI to represent characters + outside the range of the US-ASCII coded character set if this + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 8]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-9"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + representation is allowed by the scheme or by the protocol element in + which the URI is referenced. Such a definition should specify the + character encoding used to map those characters to octets prior to + being percent-encoded for the URI. + +<h4><a class="selflink" id="section-1.2.2" href="#section-1.2.2">1.2.2</a>. Separating Identification from Interaction</h4> + + A common misunderstanding of URIs is that they are only used to refer + to accessible resources. The URI itself only provides + identification; access to the resource is neither guaranteed nor + implied by the presence of a URI. Instead, any operation associated + with a URI reference is defined by the protocol element, data format + attribute, or natural language text in which it appears. + + Given a URI, a system may attempt to perform a variety of operations + on the resource, as might be characterized by words such as "access", + "update", "replace", or "find attributes". Such operations are + defined by the protocols that make use of URIs, not by this + specification. However, we do use a few general terms for describing + common operations on URIs. URI "resolution" is the process of + determining an access mechanism and the appropriate parameters + necessary to dereference a URI; this resolution may require several + iterations. To use that access mechanism to perform an action on the + URI's resource is to "dereference" the URI. + + When URIs are used within information retrieval systems to identify + sources of information, the most common form of URI dereference is + "retrieval": making use of a URI in order to retrieve a + representation of its associated resource. A "representation" is a + sequence of octets, along with representation metadata describing + those octets, that constitutes a record of the state of the resource + at the time when the representation is generated. Retrieval is + achieved by a process that might include using the URI as a cache key + to check for a locally cached representation, resolution of the URI + to determine an appropriate access mechanism (if any), and + dereference of the URI for the sake of applying a retrieval + operation. Depending on the protocols used to perform the retrieval, + additional information might be supplied about the resource (resource + metadata) and its relation to other resources. + + URI references in information retrieval systems are designed to be + late-binding: the result of an access is generally determined when it + is accessed and may vary over time or due to other aspects of the + interaction. These references are created in order to be used in the + future: what is being identified is not some specific result that was + obtained in the past, but rather some characteristic that is expected + to be true for future results. In such cases, the resource referred + to by the URI is actually a sameness of characteristics as observed + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 9]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-10"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + over time, perhaps elucidated by additional comments or assertions + made by the resource provider. + + Although many URI schemes are named after protocols, this does not + imply that use of these URIs will result in access to the resource + via the named protocol. URIs are often used simply for the sake of + identification. Even when a URI is used to retrieve a representation + of a resource, that access might be through gateways, proxies, + caches, and name resolution services that are independent of the + protocol associated with the scheme name. The resolution of some + URIs may require the use of more than one protocol (e.g., both DNS + and HTTP are typically used to access an "http" URI's origin server + when a representation isn't found in a local cache). + +<h4><a class="selflink" id="section-1.2.3" href="#section-1.2.3">1.2.3</a>. Hierarchical Identifiers</h4> + + The URI syntax is organized hierarchically, with components listed in + order of decreasing significance from left to right. For some URI + schemes, the visible hierarchy is limited to the scheme itself: + everything after the scheme component delimiter (":") is considered + opaque to URI processing. Other URI schemes make the hierarchy + explicit and visible to generic parsing algorithms. + + The generic syntax uses the slash ("/"), question mark ("?"), and + number sign ("#") characters to delimit components that are + significant to the generic parser's hierarchical interpretation of an + identifier. In addition to aiding the readability of such + identifiers through the consistent use of familiar syntax, this + uniform representation of hierarchy across naming schemes allows + scheme-independent references to be made relative to that hierarchy. + + It is often the case that a group or "tree" of documents has been + constructed to serve a common purpose, wherein the vast majority of + URI references in these documents point to resources within the tree + rather than outside it. Similarly, documents located at a particular + site are much more likely to refer to other resources at that site + than to resources at remote sites. Relative referencing of URIs + allows document trees to be partially independent of their location + and access scheme. For instance, it is possible for a single set of + hypertext documents to be simultaneously accessible and traversable + via each of the "file", "http", and "ftp" schemes if the documents + refer to each other with relative references. Furthermore, such + document trees can be moved, as a whole, without changing any of the + relative references. + + A relative reference (<a href="#section-4.2">Section 4.2</a>) refers to a resource by describing + the difference within a hierarchical name space between the reference + context and the target URI. The reference resolution algorithm, + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 10]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-11"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + presented in <a href="#section-5">Section 5</a>, defines how such a reference is transformed + to the target URI. As relative references can only be used within + the context of a hierarchical URI, designers of new URI schemes + should use a syntax consistent with the generic syntax's hierarchical + components unless there are compelling reasons to forbid relative + referencing within that scheme. + + NOTE: Previous specifications used the terms "partial URI" and + "relative URI" to denote a relative reference to a URI. As some + readers misunderstood those terms to mean that relative URIs are a + subset of URIs rather than a method of referencing URIs, this + specification simply refers to them as relative references. + + All URI references are parsed by generic syntax parsers when used. + However, because hierarchical processing has no effect on an absolute + URI used in a reference unless it contains one or more dot-segments + (complete path segments of "." or "..", as described in <a href="#section-3.3">Section 3.3</a>), + URI scheme specifications can define opaque identifiers by + disallowing use of slash characters, question mark characters, and + the URIs "scheme:." and "scheme:..". + +<h3><a class="selflink" id="section-1.3" href="#section-1.3">1.3</a>. Syntax Notation</h3> + + This specification uses the Augmented Backus-Naur Form (ABNF) + notation of [<a href="/doc/html/rfc2234">RFC2234</a>], including the following core ABNF syntax rules + defined by that specification: ALPHA (letters), CR (carriage return), + DIGIT (decimal digits), DQUOTE (double quote), HEXDIG (hexadecimal + digits), LF (line feed), and SP (space). The complete URI syntax is + collected in <a href="#appendix-A">Appendix A</a>. + +<h2><a class="selflink" id="section-2" href="#section-2">2</a>. Characters</h2> + + The URI syntax provides a method of encoding data, presumably for the + sake of identifying a resource, as a sequence of characters. The URI + characters are, in turn, frequently encoded as octets for transport + or presentation. This specification does not mandate any particular + character encoding for mapping between URI characters and the octets + used to store or transmit those characters. When a URI appears in a + protocol element, the character encoding is defined by that protocol; + without such a definition, a URI is assumed to be in the same + character encoding as the surrounding text. + + The ABNF notation defines its terminal values to be non-negative + integers (codepoints) based on the US-ASCII coded character set + [<a href="#ref-ASCII">ASCII</a>]. Because a URI is a sequence of characters, we must invert + that relation in order to understand the URI syntax. Therefore, the + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 11]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-12"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + integer values used by the ABNF must be mapped back to their + corresponding characters via US-ASCII in order to complete the syntax + rules. + + A URI is composed from a limited set of characters consisting of + digits, letters, and a few graphic symbols. A reserved subset of + those characters may be used to delimit syntax components within a + URI while the remaining characters, including both the unreserved set + and those reserved characters not acting as delimiters, define each + component's identifying data. + +<h3><a class="selflink" id="section-2.1" href="#section-2.1">2.1</a>. Percent-Encoding</h3> + + A percent-encoding mechanism is used to represent a data octet in a + component when that octet's corresponding character is outside the + allowed set or is being used as a delimiter of, or within, the + component. A percent-encoded octet is encoded as a character + triplet, consisting of the percent character "%" followed by the two + hexadecimal digits representing that octet's numeric value. For + example, "%20" is the percent-encoding for the binary octet + "00100000" (ABNF: %x20), which in US-ASCII corresponds to the space + character (SP). <a href="#section-2.4">Section 2.4</a> describes when percent-encoding and + decoding is applied. + + pct-encoded = "%" HEXDIG HEXDIG + + The uppercase hexadecimal digits 'A' through 'F' are equivalent to + the lowercase digits 'a' through 'f', respectively. If two URIs + differ only in the case of hexadecimal digits used in percent-encoded + octets, they are equivalent. For consistency, URI producers and + normalizers should use uppercase hexadecimal digits for all percent- + encodings. + +<h3><a class="selflink" id="section-2.2" href="#section-2.2">2.2</a>. Reserved Characters</h3> + + URIs include components and subcomponents that are delimited by + characters in the "reserved" set. These characters are called + "reserved" because they may (or may not) be defined as delimiters by + the generic syntax, by each scheme-specific syntax, or by the + implementation-specific syntax of a URI's dereferencing algorithm. + If data for a URI component would conflict with a reserved + character's purpose as a delimiter, then the conflicting data must be + percent-encoded before the URI is formed. + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 12]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-13"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + reserved = gen-delims / sub-delims + + gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@" + + sub-delims = "!" / "$" / "&" / "'" / "(" / ")" + / "*" / "+" / "," / ";" / "=" + + The purpose of reserved characters is to provide a set of delimiting + characters that are distinguishable from other data within a URI. + URIs that differ in the replacement of a reserved character with its + corresponding percent-encoded octet are not equivalent. Percent- + encoding a reserved character, or decoding a percent-encoded octet + that corresponds to a reserved character, will change how the URI is + interpreted by most applications. Thus, characters in the reserved + set are protected from normalization and are therefore safe to be + used by scheme-specific and producer-specific algorithms for + delimiting data subcomponents within a URI. + + A subset of the reserved characters (gen-delims) is used as + delimiters of the generic URI components described in <a href="#section-3">Section 3</a>. A + component's ABNF syntax rule will not use the reserved or gen-delims + rule names directly; instead, each syntax rule lists the characters + allowed within that component (i.e., not delimiting it), and any of + those characters that are also in the reserved set are "reserved" for + use as subcomponent delimiters within the component. Only the most + common subcomponents are defined by this specification; other + subcomponents may be defined by a URI scheme's specification, or by + the implementation-specific syntax of a URI's dereferencing + algorithm, provided that such subcomponents are delimited by + characters in the reserved set allowed within that component. + + URI producing applications should percent-encode data octets that + correspond to characters in the reserved set unless these characters + are specifically allowed by the URI scheme to represent data in that + component. If a reserved character is found in a URI component and + no delimiting role is known for that character, then it must be + interpreted as representing the data octet corresponding to that + character's encoding in US-ASCII. + +<h3><a class="selflink" id="section-2.3" href="#section-2.3">2.3</a>. Unreserved Characters</h3> + + Characters that are allowed in a URI but do not have a reserved + purpose are called unreserved. These include uppercase and lowercase + letters, decimal digits, hyphen, period, underscore, and tilde. + + unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 13]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-14"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + URIs that differ in the replacement of an unreserved character with + its corresponding percent-encoded US-ASCII octet are equivalent: they + identify the same resource. However, URI comparison implementations + do not always perform normalization prior to comparison (see <a href="#section-6">Section</a> + <a href="#section-6">6</a>). For consistency, percent-encoded octets in the ranges of ALPHA + (%41-%5A and %61-%7A), DIGIT (%30-%39), hyphen (%2D), period (%2E), + underscore (%5F), or tilde (%7E) should not be created by URI + producers and, when found in a URI, should be decoded to their + corresponding unreserved characters by URI normalizers. + +<h3><a class="selflink" id="section-2.4" href="#section-2.4">2.4</a>. When to Encode or Decode</h3> + + Under normal circumstances, the only time when octets within a URI + are percent-encoded is during the process of producing the URI from + its component parts. This is when an implementation determines which + of the reserved characters are to be used as subcomponent delimiters + and which can be safely used as data. Once produced, a URI is always + in its percent-encoded form. + + When a URI is dereferenced, the components and subcomponents + significant to the scheme-specific dereferencing process (if any) + must be parsed and separated before the percent-encoded octets within + those components can be safely decoded, as otherwise the data may be + mistaken for component delimiters. The only exception is for + percent-encoded octets corresponding to characters in the unreserved + set, which can be decoded at any time. For example, the octet + corresponding to the tilde ("~") character is often encoded as "%7E" + by older URI processing implementations; the "%7E" can be replaced by + "~" without changing its interpretation. + + Because the percent ("%") character serves as the indicator for + percent-encoded octets, it must be percent-encoded as "%25" for that + octet to be used as data within a URI. Implementations must not + percent-encode or decode the same string more than once, as decoding + an already decoded string might lead to misinterpreting a percent + data octet as the beginning of a percent-encoding, or vice versa in + the case of percent-encoding an already percent-encoded string. + +<h3><a class="selflink" id="section-2.5" href="#section-2.5">2.5</a>. Identifying Data</h3> + + URI characters provide identifying data for each of the URI + components, serving as an external interface for identification + between systems. Although the presence and nature of the URI + production interface is hidden from clients that use its URIs (and is + thus beyond the scope of the interoperability requirements defined by + this specification), it is a frequent source of confusion and errors + in the interpretation of URI character issues. Implementers have to + be aware that there are multiple character encodings involved in the + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 14]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-15"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + production and transmission of URIs: local name and data encoding, + public interface encoding, URI character encoding, data format + encoding, and protocol encoding. + + Local names, such as file system names, are stored with a local + character encoding. URI producing applications (e.g., origin + servers) will typically use the local encoding as the basis for + producing meaningful names. The URI producer will transform the + local encoding to one that is suitable for a public interface and + then transform the public interface encoding into the restricted set + of URI characters (reserved, unreserved, and percent-encodings). + Those characters are, in turn, encoded as octets to be used as a + reference within a data format (e.g., a document charset), and such + data formats are often subsequently encoded for transmission over + Internet protocols. + + For most systems, an unreserved character appearing within a URI + component is interpreted as representing the data octet corresponding + to that character's encoding in US-ASCII. Consumers of URIs assume + that the letter "X" corresponds to the octet "01011000", and even + when that assumption is incorrect, there is no harm in making it. A + system that internally provides identifiers in the form of a + different character encoding, such as EBCDIC, will generally perform + character translation of textual identifiers to UTF-8 [<a href="#ref-STD63">STD63</a>] (or + some other superset of the US-ASCII character encoding) at an + internal interface, thereby providing more meaningful identifiers + than those resulting from simply percent-encoding the original + octets. + + For example, consider an information service that provides data, + stored locally using an EBCDIC-based file system, to clients on the + Internet through an HTTP server. When an author creates a file with + the name "Laguna Beach" on that file system, the "http" URI + corresponding to that resource is expected to contain the meaningful + string "Laguna%20Beach". If, however, that server produces URIs by + using an overly simplistic raw octet mapping, then the result would + be a URI containing "%D3%81%87%A4%95%81@%C2%85%81%83%88". An + internal transcoding interface fixes this problem by transcoding the + local name to a superset of US-ASCII prior to producing the URI. + Naturally, proper interpretation of an incoming URI on such an + interface requires that percent-encoded octets be decoded (e.g., + "%20" to SP) before the reverse transcoding is applied to obtain the + local name. + + In some cases, the internal interface between a URI component and the + identifying data that it has been crafted to represent is much less + direct than a character encoding translation. For example, portions + of a URI might reflect a query on non-ASCII data, or numeric + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 15]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-16"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + coordinates on a map. Likewise, a URI scheme may define components + with additional encoding requirements that are applied prior to + forming the component and producing the URI. + + When a new URI scheme defines a component that represents textual + data consisting of characters from the Universal Character Set [<a href="#ref-UCS">UCS</a>], + the data should first be encoded as octets according to the UTF-8 + character encoding [<a href="#ref-STD63">STD63</a>]; then only those octets that do not + correspond to characters in the unreserved set should be percent- + encoded. For example, the character A would be represented as "A", + the character LATIN CAPITAL LETTER A WITH GRAVE would be represented + as "%C3%80", and the character KATAKANA LETTER A would be represented + as "%E3%82%A2". + +<h2><a class="selflink" id="section-3" href="#section-3">3</a>. Syntax Components</h2> + + The generic URI syntax consists of a hierarchical sequence of + components referred to as the scheme, authority, path, query, and + fragment. + + URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ] + + hier-part = "//" authority path-abempty + / path-absolute + / path-rootless + / path-empty + + The scheme and path components are required, though the path may be + empty (no characters). When authority is present, the path must + either be empty or begin with a slash ("/") character. When + authority is not present, the path cannot begin with two slash + characters ("//"). These restrictions result in five different ABNF + rules for a path (<a href="#section-3.3">Section 3.3</a>), only one of which will match any + given URI reference. + + The following are two example URIs and their component parts: + + foo://example.com:8042/over/there?name=ferret#nose + \_/ \______________/\_________/ \_________/ \__/ + | | | | | + scheme authority path query fragment + | _____________________|__ + / \ / \ + urn:example:animal:ferret:nose + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 16]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-17"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h3><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. Scheme</h3> + + Each URI begins with a scheme name that refers to a specification for + assigning identifiers within that scheme. As such, the URI syntax is + a federated and extensible naming system wherein each scheme's + specification may further restrict the syntax and semantics of + identifiers using that scheme. + + Scheme names consist of a sequence of characters beginning with a + letter and followed by any combination of letters, digits, plus + ("+"), period ("."), or hyphen ("-"). Although schemes are case- + insensitive, the canonical form is lowercase and documents that + specify schemes must do so with lowercase letters. An implementation + should accept uppercase letters as equivalent to lowercase in scheme + names (e.g., allow "HTTP" as well as "http") for the sake of + robustness but should only produce lowercase scheme names for + consistency. + + scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) + + Individual schemes are not specified by this document. The process + for registration of new URI schemes is defined separately by [<a href="#ref-BCP35">BCP35</a>]. + The scheme registry maintains the mapping between scheme names and + their specifications. Advice for designers of new URI schemes can be + found in [<a href="/doc/html/rfc2718">RFC2718</a>]. URI scheme specifications must define their own + syntax so that all strings matching their scheme-specific syntax will + also match the <absolute-URI> grammar, as described in <a href="#section-4.3">Section 4.3</a>. + + When presented with a URI that violates one or more scheme-specific + restrictions, the scheme-specific resolution process should flag the + reference as an error rather than ignore the unused parts; doing so + reduces the number of equivalent URIs and helps detect abuses of the + generic syntax, which might indicate that the URI has been + constructed to mislead the user (<a href="#section-7.6">Section 7.6</a>). + +<h3><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. Authority</h3> + + Many URI schemes include a hierarchical element for a naming + authority so that governance of the name space defined by the + remainder of the URI is delegated to that authority (which may, in + turn, delegate it further). The generic syntax provides a common + means for distinguishing an authority based on a registered name or + server address, along with optional port and user information. + + The authority component is preceded by a double slash ("//") and is + terminated by the next slash ("/"), question mark ("?"), or number + sign ("#") character, or by the end of the URI. + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 17]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-18"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + authority = [ userinfo "@" ] host [ ":" port ] + + URI producers and normalizers should omit the ":" delimiter that + separates host from port if the port component is empty. Some + schemes do not allow the userinfo and/or port subcomponents. + + If a URI contains an authority component, then the path component + must either be empty or begin with a slash ("/") character. Non- + validating parsers (those that merely separate a URI reference into + its major components) will often ignore the subcomponent structure of + authority, treating it as an opaque string from the double-slash to + the first terminating delimiter, until such time as the URI is + dereferenced. + +<h4><a class="selflink" id="section-3.2.1" href="#section-3.2.1">3.2.1</a>. User Information</h4> + + The userinfo subcomponent may consist of a user name and, optionally, + scheme-specific information about how to gain authorization to access + the resource. The user information, if present, is followed by a + commercial at-sign ("@") that delimits it from the host. + + userinfo = *( unreserved / pct-encoded / sub-delims / ":" ) + + Use of the format "user:password" in the userinfo field is + deprecated. Applications should not render as clear text any data + after the first colon (":") character found within a userinfo + subcomponent unless the data after the colon is the empty string + (indicating no password). Applications may choose to ignore or + reject such data when it is received as part of a reference and + should reject the storage of such data in unencrypted form. The + passing of authentication information in clear text has proven to be + a security risk in almost every case where it has been used. + + Applications that render a URI for the sake of user feedback, such as + in graphical hypertext browsing, should render userinfo in a way that + is distinguished from the rest of a URI, when feasible. Such + rendering will assist the user in cases where the userinfo has been + misleadingly crafted to look like a trusted domain name + (<a href="#section-7.6">Section 7.6</a>). + +<h4><a class="selflink" id="section-3.2.2" href="#section-3.2.2">3.2.2</a>. Host</h4> + + The host subcomponent of authority is identified by an IP literal + encapsulated within square brackets, an IPv4 address in dotted- + decimal form, or a registered name. The host subcomponent is case- + insensitive. The presence of a host subcomponent within a URI does + not imply that the scheme requires access to the given host on the + Internet. In many cases, the host syntax is used only for the sake + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 18]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-19"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + of reusing the existing registration process created and deployed for + DNS, thus obtaining a globally unique name without the cost of + deploying another registry. However, such use comes with its own + costs: domain name ownership may change over time for reasons not + anticipated by the URI producer. In other cases, the data within the + host component identifies a registered name that has nothing to do + with an Internet host. We use the name "host" for the ABNF rule + because that is its most common purpose, not its only purpose. + + host = IP-literal / IPv4address / reg-name + + The syntax rule for host is ambiguous because it does not completely + distinguish between an IPv4address and a reg-name. In order to + disambiguate the syntax, we apply the "first-match-wins" algorithm: + If host matches the rule for IPv4address, then it should be + considered an IPv4 address literal and not a reg-name. Although host + is case-insensitive, producers and normalizers should use lowercase + for registered names and hexadecimal addresses for the sake of + uniformity, while only using uppercase letters for percent-encodings. + + A host identified by an Internet Protocol literal address, version 6 + [<a href="/doc/html/rfc3513">RFC3513</a>] or later, is distinguished by enclosing the IP literal + within square brackets ("[" and "]"). This is the only place where + square bracket characters are allowed in the URI syntax. In + anticipation of future, as-yet-undefined IP literal address formats, + an implementation may use an optional version flag to indicate such a + format explicitly rather than rely on heuristic determination. + + IP-literal = "[" ( IPv6address / IPvFuture ) "]" + + IPvFuture = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" ) + + The version flag does not indicate the IP version; rather, it + indicates future versions of the literal format. As such, + implementations must not provide the version flag for the existing + IPv4 and IPv6 literal address forms described below. If a URI + containing an IP-literal that starts with "v" (case-insensitive), + indicating that the version flag is present, is dereferenced by an + application that does not know the meaning of that version flag, then + the application should return an appropriate error for "address + mechanism not supported". + + A host identified by an IPv6 literal address is represented inside + the square brackets without a preceding version flag. The ABNF + provided here is a translation of the text definition of an IPv6 + literal address provided in [<a href="/doc/html/rfc3513">RFC3513</a>]. This syntax does not support + IPv6 scoped addressing zone identifiers. + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 19]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-20"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + A 128-bit IPv6 address is divided into eight 16-bit pieces. Each + piece is represented numerically in case-insensitive hexadecimal, + using one to four hexadecimal digits (leading zeroes are permitted). + The eight encoded pieces are given most-significant first, separated + by colon characters. Optionally, the least-significant two pieces + may instead be represented in IPv4 address textual format. A + sequence of one or more consecutive zero-valued 16-bit pieces within + the address may be elided, omitting all their digits and leaving + exactly two consecutive colons in their place to mark the elision. + + IPv6address = 6( h16 ":" ) ls32 + / "::" 5( h16 ":" ) ls32 + / [ h16 ] "::" 4( h16 ":" ) ls32 + / [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32 + / [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32 + / [ *3( h16 ":" ) h16 ] "::" h16 ":" ls32 + / [ *4( h16 ":" ) h16 ] "::" ls32 + / [ *5( h16 ":" ) h16 ] "::" h16 + / [ *6( h16 ":" ) h16 ] "::" + + ls32 = ( h16 ":" h16 ) / IPv4address + ; least-significant 32 bits of address + + h16 = 1*4HEXDIG + ; 16 bits of address represented in hexadecimal + + A host identified by an IPv4 literal address is represented in + dotted-decimal notation (a sequence of four decimal numbers in the + range 0 to 255, separated by "."), as described in [<a href="/doc/html/rfc1123">RFC1123</a>] by + reference to [<a href="/doc/html/rfc0952">RFC0952</a>]. Note that other forms of dotted notation may + be interpreted on some platforms, as described in <a href="#section-7.4">Section 7.4</a>, but + only the dotted-decimal form of four octets is allowed by this + grammar. + + IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet + + dec-octet = DIGIT ; 0-9 + / %x31-39 DIGIT ; 10-99 + / "1" 2DIGIT ; 100-199 + / "2" %x30-34 DIGIT ; 200-249 + / "25" %x30-35 ; 250-255 + + A host identified by a registered name is a sequence of characters + usually intended for lookup within a locally defined host or service + name registry, though the URI's scheme-specific semantics may require + that a specific registry (or fixed name table) be used instead. The + most common name registry mechanism is the Domain Name System (DNS). + A registered name intended for lookup in the DNS uses the syntax + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 20]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-21"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + defined in <a href="/doc/html/rfc1034#section-3.5">Section 3.5 of [RFC1034]</a> and <a href="/doc/html/rfc1123#section-2.1">Section 2.1 of [RFC1123]</a>. + Such a name consists of a sequence of domain labels separated by ".", + each domain label starting and ending with an alphanumeric character + and possibly also containing "-" characters. The rightmost domain + label of a fully qualified domain name in DNS may be followed by a + single "." and should be if it is necessary to distinguish between + the complete domain name and some local domain. + + reg-name = *( unreserved / pct-encoded / sub-delims ) + + If the URI scheme defines a default for host, then that default + applies when the host subcomponent is undefined or when the + registered name is empty (zero length). For example, the "file" URI + scheme is defined so that no authority, an empty host, and + "localhost" all mean the end-user's machine, whereas the "http" + scheme considers a missing authority or empty host invalid. + + This specification does not mandate a particular registered name + lookup technology and therefore does not restrict the syntax of reg- + name beyond what is necessary for interoperability. Instead, it + delegates the issue of registered name syntax conformance to the + operating system of each application performing URI resolution, and + that operating system decides what it will allow for the purpose of + host identification. A URI resolution implementation might use DNS, + host tables, yellow pages, NetInfo, WINS, or any other system for + lookup of registered names. However, a globally scoped naming + system, such as DNS fully qualified domain names, is necessary for + URIs intended to have global scope. URI producers should use names + that conform to the DNS syntax, even when use of DNS is not + immediately apparent, and should limit these names to no more than + 255 characters in length. + + The reg-name syntax allows percent-encoded octets in order to + represent non-ASCII registered names in a uniform way that is + independent of the underlying name resolution technology. Non-ASCII + characters must first be encoded according to UTF-8 [<a href="#ref-STD63">STD63</a>], and then + each octet of the corresponding UTF-8 sequence must be percent- + encoded to be represented as URI characters. URI producing + applications must not use percent-encoding in host unless it is used + to represent a UTF-8 character sequence. When a non-ASCII registered + name represents an internationalized domain name intended for + resolution via the DNS, the name must be transformed to the IDNA + encoding [<a href="/doc/html/rfc3490">RFC3490</a>] prior to name lookup. URI producers should + provide these registered names in the IDNA encoding, rather than a + percent-encoding, if they wish to maximize interoperability with + legacy URI resolvers. + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 21]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-22"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h4><a class="selflink" id="section-3.2.3" href="#section-3.2.3">3.2.3</a>. Port</h4> + + The port subcomponent of authority is designated by an optional port + number in decimal following the host and delimited from it by a + single colon (":") character. + + port = *DIGIT + + A scheme may define a default port. For example, the "http" scheme + defines a default port of "80", corresponding to its reserved TCP + port number. The type of port designated by the port number (e.g., + TCP, UDP, SCTP) is defined by the URI scheme. URI producers and + normalizers should omit the port component and its ":" delimiter if + port is empty or if its value would be the same as that of the + scheme's default. + +<h3><a class="selflink" id="section-3.3" href="#section-3.3">3.3</a>. Path</h3> + + The path component contains data, usually organized in hierarchical + form, that, along with data in the non-hierarchical query component + (<a href="#section-3.4">Section 3.4</a>), serves to identify a resource within the scope of the + URI's scheme and naming authority (if any). The path is terminated + by the first question mark ("?") or number sign ("#") character, or + by the end of the URI. + + If a URI contains an authority component, then the path component + must either be empty or begin with a slash ("/") character. If a URI + does not contain an authority component, then the path cannot begin + with two slash characters ("//"). In addition, a URI reference + (<a href="#section-4.1">Section 4.1</a>) may be a relative-path reference, in which case the + first path segment cannot contain a colon (":") character. The ABNF + requires five separate rules to disambiguate these cases, only one of + which will match the path substring within a given URI reference. We + use the generic term "path component" to describe the URI substring + matched by the parser to one of these rules. + + path = path-abempty ; begins with "/" or is empty + / path-absolute ; begins with "/" but not "//" + / path-noscheme ; begins with a non-colon segment + / path-rootless ; begins with a segment + / path-empty ; zero characters + + path-abempty = *( "/" segment ) + path-absolute = "/" [ segment-nz *( "/" segment ) ] + path-noscheme = segment-nz-nc *( "/" segment ) + path-rootless = segment-nz *( "/" segment ) + path-empty = 0<pchar> + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 22]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-23"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + segment = *pchar + segment-nz = 1*pchar + segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" ) + ; non-zero-length segment without any colon ":" + + pchar = unreserved / pct-encoded / sub-delims / ":" / "@" + + A path consists of a sequence of path segments separated by a slash + ("/") character. A path is always defined for a URI, though the + defined path may be empty (zero length). Use of the slash character + to indicate hierarchy is only required when a URI will be used as the + context for relative references. For example, the URI + <mailto:fred@example.com> has a path of "fred@example.com", whereas + the URI <foo://info.example.com?fred> has an empty path. + + The path segments "." and "..", also known as dot-segments, are + defined for relative reference within the path name hierarchy. They + are intended for use at the beginning of a relative-path reference + (<a href="#section-4.2">Section 4.2</a>) to indicate relative position within the hierarchical + tree of names. This is similar to their role within some operating + systems' file directory structures to indicate the current directory + and parent directory, respectively. However, unlike in a file + system, these dot-segments are only interpreted within the URI path + hierarchy and are removed as part of the resolution process (<a href="#section-5.2">Section</a> + <a href="#section-5.2">5.2</a>). + + Aside from dot-segments in hierarchical paths, a path segment is + considered opaque by the generic syntax. URI producing applications + often use the reserved characters allowed in a segment to delimit + scheme-specific or dereference-handler-specific subcomponents. For + example, the semicolon (";") and equals ("=") reserved characters are + often used to delimit parameters and parameter values applicable to + that segment. The comma (",") reserved character is often used for + similar purposes. For example, one URI producer might use a segment + such as "name;v=1.1" to indicate a reference to version 1.1 of + "name", whereas another might use a segment such as "name,1.1" to + indicate the same. Parameter types may be defined by scheme-specific + semantics, but in most cases the syntax of a parameter is specific to + the implementation of the URI's dereferencing algorithm. + +<h3><a class="selflink" id="section-3.4" href="#section-3.4">3.4</a>. Query</h3> + + The query component contains non-hierarchical data that, along with + data in the path component (<a href="#section-3.3">Section 3.3</a>), serves to identify a + resource within the scope of the URI's scheme and naming authority + (if any). The query component is indicated by the first question + mark ("?") character and terminated by a number sign ("#") character + or by the end of the URI. + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 23]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-24"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + query = *( pchar / "/" / "?" ) + + The characters slash ("/") and question mark ("?") may represent data + within the query component. Beware that some older, erroneous + implementations may not handle such data correctly when it is used as + the base URI for relative references (<a href="#section-5.1">Section 5.1</a>), apparently + because they fail to distinguish query data from path data when + looking for hierarchical separators. However, as query components + are often used to carry identifying information in the form of + "key=value" pairs and one frequently used value is a reference to + another URI, it is sometimes better for usability to avoid percent- + encoding those characters. + +<h3><a class="selflink" id="section-3.5" href="#section-3.5">3.5</a>. Fragment</h3> + + The fragment identifier component of a URI allows indirect + identification of a secondary resource by reference to a primary + resource and additional identifying information. The identified + secondary resource may be some portion or subset of the primary + resource, some view on representations of the primary resource, or + some other resource defined or described by those representations. A + fragment identifier component is indicated by the presence of a + number sign ("#") character and terminated by the end of the URI. + + fragment = *( pchar / "/" / "?" ) + + The semantics of a fragment identifier are defined by the set of + representations that might result from a retrieval action on the + primary resource. The fragment's format and resolution is therefore + dependent on the media type [<a href="/doc/html/rfc2046">RFC2046</a>] of a potentially retrieved + representation, even though such a retrieval is only performed if the + URI is dereferenced. If no such representation exists, then the + semantics of the fragment are considered unknown and are effectively + unconstrained. Fragment identifier semantics are independent of the + URI scheme and thus cannot be redefined by scheme specifications. + + Individual media types may define their own restrictions on or + structures within the fragment identifier syntax for specifying + different types of subsets, views, or external references that are + identifiable as secondary resources by that media type. If the + primary resource has multiple representations, as is often the case + for resources whose representation is selected based on attributes of + the retrieval request (a.k.a., content negotiation), then whatever is + identified by the fragment should be consistent across all of those + representations. Each representation should either define the + fragment so that it corresponds to the same secondary resource, + regardless of how it is represented, or should leave the fragment + undefined (i.e., not found). + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 24]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-25"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + As with any URI, use of a fragment identifier component does not + imply that a retrieval action will take place. A URI with a fragment + identifier may be used to refer to the secondary resource without any + implication that the primary resource is accessible or will ever be + accessed. + + Fragment identifiers have a special role in information retrieval + systems as the primary form of client-side indirect referencing, + allowing an author to specifically identify aspects of an existing + resource that are only indirectly provided by the resource owner. As + such, the fragment identifier is not used in the scheme-specific + processing of a URI; instead, the fragment identifier is separated + from the rest of the URI prior to a dereference, and thus the + identifying information within the fragment itself is dereferenced + solely by the user agent, regardless of the URI scheme. Although + this separate handling is often perceived to be a loss of + information, particularly for accurate redirection of references as + resources move over time, it also serves to prevent information + providers from denying reference authors the right to refer to + information within a resource selectively. Indirect referencing also + provides additional flexibility and extensibility to systems that use + URIs, as new media types are easier to define and deploy than new + schemes of identification. + + The characters slash ("/") and question mark ("?") are allowed to + represent data within the fragment identifier. Beware that some + older, erroneous implementations may not handle this data correctly + when it is used as the base URI for relative references (<a href="#section-5.1">Section</a> + <a href="#section-5.1">5.1</a>). + +<h2><a class="selflink" id="section-4" href="#section-4">4</a>. Usage</h2> + + When applications make reference to a URI, they do not always use the + full form of reference defined by the "URI" syntax rule. To save + space and take advantage of hierarchical locality, many Internet + protocol elements and media type formats allow an abbreviation of a + URI, whereas others restrict the syntax to a particular form of URI. + We define the most common forms of reference syntax in this + specification because they impact and depend upon the design of the + generic syntax, requiring a uniform parsing algorithm in order to be + interpreted consistently. + +<h3><a class="selflink" id="section-4.1" href="#section-4.1">4.1</a>. URI Reference</h3> + + URI-reference is used to denote the most common usage of a resource + identifier. + + URI-reference = URI / relative-ref + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 25]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-26"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + A URI-reference is either a URI or a relative reference. If the + URI-reference's prefix does not match the syntax of a scheme followed + by its colon separator, then the URI-reference is a relative + reference. + + A URI-reference is typically parsed first into the five URI + components, in order to determine what components are present and + whether the reference is relative. Then, each component is parsed + for its subparts and their validation. The ABNF of URI-reference, + along with the "first-match-wins" disambiguation rule, is sufficient + to define a validating parser for the generic syntax. Readers + familiar with regular expressions should see <a href="#appendix-B">Appendix B</a> for an + example of a non-validating URI-reference parser that will take any + given string and extract the URI components. + +<h3><a class="selflink" id="section-4.2" href="#section-4.2">4.2</a>. Relative Reference</h3> + + A relative reference takes advantage of the hierarchical syntax + (<a href="#section-1.2.3">Section 1.2.3</a>) to express a URI reference relative to the name space + of another hierarchical URI. + + relative-ref = relative-part [ "?" query ] [ "#" fragment ] + + relative-part = "//" authority path-abempty + / path-absolute + / path-noscheme + / path-empty + + The URI referred to by a relative reference, also known as the target + URI, is obtained by applying the reference resolution algorithm of + <a href="#section-5">Section 5</a>. + + A relative reference that begins with two slash characters is termed + a network-path reference; such references are rarely used. A + relative reference that begins with a single slash character is + termed an absolute-path reference. A relative reference that does + not begin with a slash character is termed a relative-path reference. + + A path segment that contains a colon character (e.g., "this:that") + cannot be used as the first segment of a relative-path reference, as + it would be mistaken for a scheme name. Such a segment must be + preceded by a dot-segment (e.g., "./this:that") to make a relative- + path reference. + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 26]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-27"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h3><a class="selflink" id="section-4.3" href="#section-4.3">4.3</a>. Absolute URI</h3> + + Some protocol elements allow only the absolute form of a URI without + a fragment identifier. For example, defining a base URI for later + use by relative references calls for an absolute-URI syntax rule that + does not allow a fragment. + + absolute-URI = scheme ":" hier-part [ "?" query ] + + URI scheme specifications must define their own syntax so that all + strings matching their scheme-specific syntax will also match the + <absolute-URI> grammar. Scheme specifications will not define + fragment identifier syntax or usage, regardless of its applicability + to resources identifiable via that scheme, as fragment identification + is orthogonal to scheme definition. However, scheme specifications + are encouraged to include a wide range of examples, including + examples that show use of the scheme's URIs with fragment identifiers + when such usage is appropriate. + +<h3><a class="selflink" id="section-4.4" href="#section-4.4">4.4</a>. Same-Document Reference</h3> + + When a URI reference refers to a URI that is, aside from its fragment + component (if any), identical to the base URI (<a href="#section-5.1">Section 5.1</a>), that + reference is called a "same-document" reference. The most frequent + examples of same-document references are relative references that are + empty or include only the number sign ("#") separator followed by a + fragment identifier. + + When a same-document reference is dereferenced for a retrieval + action, the target of that reference is defined to be within the same + entity (representation, document, or message) as the reference; + therefore, a dereference should not result in a new retrieval action. + + Normalization of the base and target URIs prior to their comparison, + as described in Sections <a href="#section-6.2.2">6.2.2</a> and <a href="#section-6.2.3">6.2.3</a>, is allowed but rarely + performed in practice. Normalization may increase the set of same- + document references, which may be of benefit to some caching + applications. As such, reference authors should not assume that a + slightly different, though equivalent, reference URI will (or will + not) be interpreted as a same-document reference by any given + application. + +<h3><a class="selflink" id="section-4.5" href="#section-4.5">4.5</a>. Suffix Reference</h3> + + The URI syntax is designed for unambiguous reference to resources and + extensibility via the URI scheme. However, as URI identification and + usage have become commonplace, traditional media (television, radio, + newspapers, billboards, etc.) have increasingly used a suffix of the + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 27]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-28"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + URI as a reference, consisting of only the authority and path + portions of the URI, such as + + www.w3.org/Addressing/ + + or simply a DNS registered name on its own. Such references are + primarily intended for human interpretation rather than for machines, + with the assumption that context-based heuristics are sufficient to + complete the URI (e.g., most registered names beginning with "www" + are likely to have a URI prefix of "http://"). Although there is no + standard set of heuristics for disambiguating a URI suffix, many + client implementations allow them to be entered by the user and + heuristically resolved. + + Although this practice of using suffix references is common, it + should be avoided whenever possible and should never be used in + situations where long-term references are expected. The heuristics + noted above will change over time, particularly when a new URI scheme + becomes popular, and are often incorrect when used out of context. + Furthermore, they can lead to security issues along the lines of + those described in [<a href="/doc/html/rfc1535">RFC1535</a>]. + + As a URI suffix has the same syntax as a relative-path reference, a + suffix reference cannot be used in contexts where a relative + reference is expected. As a result, suffix references are limited to + places where there is no defined base URI, such as dialog boxes and + off-line advertisements. + +<h2><a class="selflink" id="section-5" href="#section-5">5</a>. Reference Resolution</h2> + + This section defines the process of resolving a URI reference within + a context that allows relative references so that the result is a + string matching the <URI> syntax rule of <a href="#section-3">Section 3</a>. + +<h3><a class="selflink" id="section-5.1" href="#section-5.1">5.1</a>. Establishing a Base URI</h3> + + The term "relative" implies that a "base URI" exists against which + the relative reference is applied. Aside from fragment-only + references (<a href="#section-4.4">Section 4.4</a>), relative references are only usable when a + base URI is known. A base URI must be established by the parser + prior to parsing URI references that might be relative. A base URI + must conform to the <absolute-URI> syntax rule (<a href="#section-4.3">Section 4.3</a>). If the + base URI is obtained from a URI reference, then that reference must + be converted to absolute form and stripped of any fragment component + prior to its use as a base URI. + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 28]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-29"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + The base URI of a reference can be established in one of four ways, + discussed below in order of precedence. The order of precedence can + be thought of in terms of layers, where the innermost defined base + URI has the highest precedence. This can be visualized graphically + as follows: + + .----------------------------------------------------------. + | .----------------------------------------------------. | + | | .----------------------------------------------. | | + | | | .----------------------------------------. | | | + | | | | .----------------------------------. | | | | + | | | | | <relative-reference> | | | | | + | | | | `----------------------------------' | | | | + | | | | (5.1.1) Base URI embedded in content | | | | + | | | `----------------------------------------' | | | + | | | (5.1.2) Base URI of the encapsulating entity | | | + | | | (message, representation, or none) | | | + | | `----------------------------------------------' | | + | | (5.1.3) URI used to retrieve the entity | | + | `----------------------------------------------------' | + | (5.1.4) Default Base URI (application-dependent) | + `----------------------------------------------------------' + +<h4><a class="selflink" id="section-5.1.1" href="#section-5.1.1">5.1.1</a>. Base URI Embedded in Content</h4> + + Within certain media types, a base URI for relative references can be + embedded within the content itself so that it can be readily obtained + by a parser. This can be useful for descriptive documents, such as + tables of contents, which may be transmitted to others through + protocols other than their usual retrieval context (e.g., email or + USENET news). + + It is beyond the scope of this specification to specify how, for each + media type, a base URI can be embedded. The appropriate syntax, when + available, is described by the data format specification associated + with each media type. + +<h4><a class="selflink" id="section-5.1.2" href="#section-5.1.2">5.1.2</a>. Base URI from the Encapsulating Entity</h4> + + If no base URI is embedded, the base URI is defined by the + representation's retrieval context. For a document that is enclosed + within another entity, such as a message or archive, the retrieval + context is that entity. Thus, the default base URI of a + representation is the base URI of the entity in which the + representation is encapsulated. + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 29]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-30"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + A mechanism for embedding a base URI within MIME container types + (e.g., the message and multipart types) is defined by MHTML + [<a href="/doc/html/rfc2557">RFC2557</a>]. Protocols that do not use the MIME message header syntax, + but that do allow some form of tagged metadata to be included within + messages, may define their own syntax for defining a base URI as part + of a message. + +<h4><a class="selflink" id="section-5.1.3" href="#section-5.1.3">5.1.3</a>. Base URI from the Retrieval URI</h4> + + If no base URI is embedded and the representation is not encapsulated + within some other entity, then, if a URI was used to retrieve the + representation, that URI shall be considered the base URI. Note that + if the retrieval was the result of a redirected request, the last URI + used (i.e., the URI that resulted in the actual retrieval of the + representation) is the base URI. + +<h4><a class="selflink" id="section-5.1.4" href="#section-5.1.4">5.1.4</a>. Default Base URI</h4> + + If none of the conditions described above apply, then the base URI is + defined by the context of the application. As this definition is + necessarily application-dependent, failing to define a base URI by + using one of the other methods may result in the same content being + interpreted differently by different types of applications. + + A sender of a representation containing relative references is + responsible for ensuring that a base URI for those references can be + established. Aside from fragment-only references, relative + references can only be used reliably in situations where the base URI + is well defined. + +<h3><a class="selflink" id="section-5.2" href="#section-5.2">5.2</a>. Relative Resolution</h3> + + This section describes an algorithm for converting a URI reference + that might be relative to a given base URI into the parsed components + of the reference's target. The components can then be recomposed, as + described in <a href="#section-5.3">Section 5.3</a>, to form the target URI. This algorithm + provides definitive results that can be used to test the output of + other implementations. Applications may implement relative reference + resolution by using some other algorithm, provided that the results + match what would be given by this one. + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 30]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-31"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h4><a class="selflink" id="section-5.2.1" href="#section-5.2.1">5.2.1</a>. Pre-parse the Base URI</h4> + + The base URI (Base) is established according to the procedure of + <a href="#section-5.1">Section 5.1</a> and parsed into the five main components described in + <a href="#section-3">Section 3</a>. Note that only the scheme component is required to be + present in a base URI; the other components may be empty or + undefined. A component is undefined if its associated delimiter does + not appear in the URI reference; the path component is never + undefined, though it may be empty. + + Normalization of the base URI, as described in Sections <a href="#section-6.2.2">6.2.2</a> and + 6.2.3, is optional. A URI reference must be transformed to its + target URI before it can be normalized. + +<h4><a class="selflink" id="section-5.2.2" href="#section-5.2.2">5.2.2</a>. Transform References</h4> + + For each URI reference (R), the following pseudocode describes an + algorithm for transforming R into its target URI (T): + + -- The URI reference is parsed into the five URI components + -- + (R.scheme, R.authority, R.path, R.query, R.fragment) = parse(R); + + -- A non-strict parser may ignore a scheme in the reference + -- if it is identical to the base URI's scheme. + -- + if ((not strict) and (R.scheme == Base.scheme)) then + undefine(R.scheme); + endif; + + + + + + + + + + + + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 31]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-32"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + if defined(R.scheme) then + T.scheme = R.scheme; + T.authority = R.authority; + T.path = remove_dot_segments(R.path); + T.query = R.query; + else + if defined(R.authority) then + T.authority = R.authority; + T.path = remove_dot_segments(R.path); + T.query = R.query; + else + if (R.path == "") then + T.path = Base.path; + if defined(R.query) then + T.query = R.query; + else + T.query = Base.query; + endif; + else + if (R.path starts-with "/") then + T.path = remove_dot_segments(R.path); + else + T.path = merge(Base.path, R.path); + T.path = remove_dot_segments(T.path); + endif; + T.query = R.query; + endif; + T.authority = Base.authority; + endif; + T.scheme = Base.scheme; + endif; + + T.fragment = R.fragment; + +<h4><a class="selflink" id="section-5.2.3" href="#section-5.2.3">5.2.3</a>. Merge Paths</h4> + + The pseudocode above refers to a "merge" routine for merging a + relative-path reference with the path of the base URI. This is + accomplished as follows: + + o If the base URI has a defined authority component and an empty + path, then return a string consisting of "/" concatenated with the + reference's path; otherwise, + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 32]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-33"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + o return a string consisting of the reference's path component + appended to all but the last segment of the base URI's path (i.e., + excluding any characters after the right-most "/" in the base URI + path, or excluding the entire base URI path if it does not contain + any "/" characters). + +<h4><a class="selflink" id="section-5.2.4" href="#section-5.2.4">5.2.4</a>. Remove Dot Segments</h4> + + The pseudocode also refers to a "remove_dot_segments" routine for + interpreting and removing the special "." and ".." complete path + segments from a referenced path. This is done after the path is + extracted from a reference, whether or not the path was relative, in + order to remove any invalid or extraneous dot-segments prior to + forming the target URI. Although there are many ways to accomplish + this removal process, we describe a simple method using two string + buffers. + + 1. The input buffer is initialized with the now-appended path + components and the output buffer is initialized to the empty + string. + + 2. While the input buffer is not empty, loop as follows: + + A. If the input buffer begins with a prefix of "../" or "./", + then remove that prefix from the input buffer; otherwise, + + B. if the input buffer begins with a prefix of "/./" or "/.", + where "." is a complete path segment, then replace that + prefix with "/" in the input buffer; otherwise, + + C. if the input buffer begins with a prefix of "/../" or "/..", + where ".." is a complete path segment, then replace that + prefix with "/" in the input buffer and remove the last + segment and its preceding "/" (if any) from the output + buffer; otherwise, + + D. if the input buffer consists only of "." or "..", then remove + that from the input buffer; otherwise, + + E. move the first path segment in the input buffer to the end of + the output buffer, including the initial "/" character (if + any) and any subsequent characters up to, but not including, + the next "/" character or the end of the input buffer. + + 3. Finally, the output buffer is returned as the result of + remove_dot_segments. + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 33]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-34"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + Note that dot-segments are intended for use in URI references to + express an identifier relative to the hierarchy of names in the base + URI. The remove_dot_segments algorithm respects that hierarchy by + removing extra dot-segments rather than treat them as an error or + leaving them to be misinterpreted by dereference implementations. + + The following illustrates how the above steps are applied for two + examples of merged paths, showing the state of the two buffers after + each step. + + STEP OUTPUT BUFFER INPUT BUFFER + + 1 : /a/b/c/./../../g + 2E: /a /b/c/./../../g + 2E: /a/b /c/./../../g + 2E: /a/b/c /./../../g + 2B: /a/b/c /../../g + 2C: /a/b /../g + 2C: /a /g + 2E: /a/g + + STEP OUTPUT BUFFER INPUT BUFFER + + <a href="#section-1">1</a> : mid/content=5/../6 + 2E: mid /content=5/../6 + 2E: mid/content=5 /../6 + 2C: mid /6 + 2E: mid/6 + + Some applications may find it more efficient to implement the + remove_dot_segments algorithm by using two segment stacks rather than + strings. + + Note: Beware that some older, erroneous implementations will fail + to separate a reference's query component from its path component + prior to merging the base and reference paths, resulting in an + interoperability failure if the query component contains the + strings "/../" or "/./". + + + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 34]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-35"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h3><a class="selflink" id="section-5.3" href="#section-5.3">5.3</a>. Component Recomposition</h3> + + Parsed URI components can be recomposed to obtain the corresponding + URI reference string. Using pseudocode, this would be: + + result = "" + + if defined(scheme) then + append scheme to result; + append ":" to result; + endif; + + if defined(authority) then + append "//" to result; + append authority to result; + endif; + + append path to result; + + if defined(query) then + append "?" to result; + append query to result; + endif; + + if defined(fragment) then + append "#" to result; + append fragment to result; + endif; + + return result; + + Note that we are careful to preserve the distinction between a + component that is undefined, meaning that its separator was not + present in the reference, and a component that is empty, meaning that + the separator was present and was immediately followed by the next + component separator or the end of the reference. + +<h3><a class="selflink" id="section-5.4" href="#section-5.4">5.4</a>. Reference Resolution Examples</h3> + + Within a representation with a well defined base URI of + + http://a/b/c/d;p?q + + a relative reference is transformed to its target URI as follows. + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 35]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-36"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h4><a class="selflink" id="section-5.4.1" href="#section-5.4.1">5.4.1</a>. Normal Examples</h4> + + "g:h" = "g:h" + "g" = "http://a/b/c/g" + "./g" = "http://a/b/c/g" + "g/" = "http://a/b/c/g/" + "/g" = "http://a/g" + "//g" = "http://g" + "?y" = "http://a/b/c/d;p?y" + "g?y" = "http://a/b/c/g?y" + "#s" = "http://a/b/c/d;p?q#s" + "g#s" = "http://a/b/c/g#s" + "g?y#s" = "http://a/b/c/g?y#s" + ";x" = "http://a/b/c/;x" + "g;x" = "http://a/b/c/g;x" + "g;x?y#s" = "http://a/b/c/g;x?y#s" + "" = "http://a/b/c/d;p?q" + "." = "http://a/b/c/" + "./" = "http://a/b/c/" + ".." = "http://a/b/" + "../" = "http://a/b/" + "../g" = "http://a/b/g" + "../.." = "http://a/" + "../../" = "http://a/" + "../../g" = "http://a/g" + +<h4><a class="selflink" id="section-5.4.2" href="#section-5.4.2">5.4.2</a>. Abnormal Examples</h4> + + Although the following abnormal examples are unlikely to occur in + normal practice, all URI parsers should be capable of resolving them + consistently. Each example uses the same base as that above. + + Parsers must be careful in handling cases where there are more ".." + segments in a relative-path reference than there are hierarchical + levels in the base URI's path. Note that the ".." syntax cannot be + used to change the authority component of a URI. + + "../../../g" = "http://a/g" + "../../../../g" = "http://a/g" + + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 36]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-37"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + Similarly, parsers must remove the dot-segments "." and ".." when + they are complete components of a path, but not when they are only + part of a segment. + + "/./g" = "http://a/g" + "/../g" = "http://a/g" + "g." = "http://a/b/c/g." + ".g" = "http://a/b/c/.g" + "g.." = "http://a/b/c/g.." + "..g" = "http://a/b/c/..g" + + Less likely are cases where the relative reference uses unnecessary + or nonsensical forms of the "." and ".." complete path segments. + + "./../g" = "http://a/b/g" + "./g/." = "http://a/b/c/g/" + "g/./h" = "http://a/b/c/g/h" + "g/../h" = "http://a/b/c/h" + "g;x=1/./y" = "http://a/b/c/g;x=1/y" + "g;x=1/../y" = "http://a/b/c/y" + + Some applications fail to separate the reference's query and/or + fragment components from the path component before merging it with + the base path and removing dot-segments. This error is rarely + noticed, as typical usage of a fragment never includes the hierarchy + ("/") character and the query component is not normally used within + relative references. + + "g?y/./x" = "http://a/b/c/g?y/./x" + "g?y/../x" = "http://a/b/c/g?y/../x" + "g#s/./x" = "http://a/b/c/g#s/./x" + "g#s/../x" = "http://a/b/c/g#s/../x" + + Some parsers allow the scheme name to be present in a relative + reference if it is the same as the base URI scheme. This is + considered to be a loophole in prior specifications of partial URI + [<a href="/doc/html/rfc1630">RFC1630</a>]. Its use should be avoided but is allowed for backward + compatibility. + + "http:g" = "http:g" ; for strict parsers + / "http://a/b/c/g" ; for backward compatibility + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 37]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-38"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h2><a class="selflink" id="section-6" href="#section-6">6</a>. Normalization and Comparison</h2> + + One of the most common operations on URIs is simple comparison: + determining whether two URIs are equivalent without using the URIs to + access their respective resource(s). A comparison is performed every + time a response cache is accessed, a browser checks its history to + color a link, or an XML parser processes tags within a namespace. + Extensive normalization prior to comparison of URIs is often used by + spiders and indexing engines to prune a search space or to reduce + duplication of request actions and response storage. + + URI comparison is performed for some particular purpose. Protocols + or implementations that compare URIs for different purposes will + often be subject to differing design trade-offs in regards to how + much effort should be spent in reducing aliased identifiers. This + section describes various methods that may be used to compare URIs, + the trade-offs between them, and the types of applications that might + use them. + +<h3><a class="selflink" id="section-6.1" href="#section-6.1">6.1</a>. Equivalence</h3> + + Because URIs exist to identify resources, presumably they should be + considered equivalent when they identify the same resource. However, + this definition of equivalence is not of much practical use, as there + is no way for an implementation to compare two resources unless it + has full knowledge or control of them. For this reason, + determination of equivalence or difference of URIs is based on string + comparison, perhaps augmented by reference to additional rules + provided by URI scheme definitions. We use the terms "different" and + "equivalent" to describe the possible outcomes of such comparisons, + but there are many application-dependent versions of equivalence. + + Even though it is possible to determine that two URIs are equivalent, + URI comparison is not sufficient to determine whether two URIs + identify different resources. For example, an owner of two different + domain names could decide to serve the same resource from both, + resulting in two different URIs. Therefore, comparison methods are + designed to minimize false negatives while strictly avoiding false + positives. + + In testing for equivalence, applications should not directly compare + relative references; the references should be converted to their + respective target URIs before comparison. When URIs are compared to + select (or avoid) a network action, such as retrieval of a + representation, fragment components (if any) should be excluded from + the comparison. + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 38]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-39"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h3><a class="selflink" id="section-6.2" href="#section-6.2">6.2</a>. Comparison Ladder</h3> + + A variety of methods are used in practice to test URI equivalence. + These methods fall into a range, distinguished by the amount of + processing required and the degree to which the probability of false + negatives is reduced. As noted above, false negatives cannot be + eliminated. In practice, their probability can be reduced, but this + reduction requires more processing and is not cost-effective for all + applications. + + If this range of comparison practices is considered as a ladder, the + following discussion will climb the ladder, starting with practices + that are cheap but have a relatively higher chance of producing false + negatives, and proceeding to those that have higher computational + cost and lower risk of false negatives. + +<h4><a class="selflink" id="section-6.2.1" href="#section-6.2.1">6.2.1</a>. Simple String Comparison</h4> + + If two URIs, when considered as character strings, are identical, + then it is safe to conclude that they are equivalent. This type of + equivalence test has very low computational cost and is in wide use + in a variety of applications, particularly in the domain of parsing. + + Testing strings for equivalence requires some basic precautions. + This procedure is often referred to as "bit-for-bit" or + "byte-for-byte" comparison, which is potentially misleading. Testing + strings for equality is normally based on pair comparison of the + characters that make up the strings, starting from the first and + proceeding until both strings are exhausted and all characters are + found to be equal, until a pair of characters compares unequal, or + until one of the strings is exhausted before the other. + + This character comparison requires that each pair of characters be + put in comparable form. For example, should one URI be stored in a + byte array in EBCDIC encoding and the second in a Java String object + (UTF-16), bit-for-bit comparisons applied naively will produce + errors. It is better to speak of equality on a character-for- + character basis rather than on a byte-for-byte or bit-for-bit basis. + In practical terms, character-by-character comparisons should be done + codepoint-by-codepoint after conversion to a common character + encoding. + + False negatives are caused by the production and use of URI aliases. + Unnecessary aliases can be reduced, regardless of the comparison + method, by consistently providing URI references in an already- + normalized form (i.e., a form identical to what would be produced + after normalization is applied, as described below). + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 39]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-40"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + Protocols and data formats often limit some URI comparisons to simple + string comparison, based on the theory that people and + implementations will, in their own best interest, be consistent in + providing URI references, or at least consistent enough to negate any + efficiency that might be obtained from further normalization. + +<h4><a class="selflink" id="section-6.2.2" href="#section-6.2.2">6.2.2</a>. Syntax-Based Normalization</h4> + + Implementations may use logic based on the definitions provided by + this specification to reduce the probability of false negatives. + This processing is moderately higher in cost than character-for- + character string comparison. For example, an application using this + approach could reasonably consider the following two URIs equivalent: + + example://a/b/c/%7Bfoo%7D + eXAMPLE://a/./b/../b/%63/%7bfoo%7d + + Web user agents, such as browsers, typically apply this type of URI + normalization when determining whether a cached response is + available. Syntax-based normalization includes such techniques as + case normalization, percent-encoding normalization, and removal of + dot-segments. + +<h5><a class="selflink" id="section-6.2.2.1" href="#section-6.2.2.1">6.2.2.1</a>. Case Normalization</h5> + + For all URIs, the hexadecimal digits within a percent-encoding + triplet (e.g., "%3a" versus "%3A") are case-insensitive and therefore + should be normalized to use uppercase letters for the digits A-F. + + When a URI uses components of the generic syntax, the component + syntax equivalence rules always apply; namely, that the scheme and + host are case-insensitive and therefore should be normalized to + lowercase. For example, the URI <HTTP://www.EXAMPLE.com/> is + equivalent to <http://www.example.com/>. The other generic syntax + components are assumed to be case-sensitive unless specifically + defined otherwise by the scheme (see <a href="#section-6.2.3">Section 6.2.3</a>). + +<h5><a class="selflink" id="section-6.2.2.2" href="#section-6.2.2.2">6.2.2.2</a>. Percent-Encoding Normalization</h5> + + The percent-encoding mechanism (<a href="#section-2.1">Section 2.1</a>) is a frequent source of + variance among otherwise identical URIs. In addition to the case + normalization issue noted above, some URI producers percent-encode + octets that do not require percent-encoding, resulting in URIs that + are equivalent to their non-encoded counterparts. These URIs should + be normalized by decoding any percent-encoded octet that corresponds + to an unreserved character, as described in <a href="#section-2.3">Section 2.3</a>. + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 40]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-41"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h5><a class="selflink" id="section-6.2.2.3" href="#section-6.2.2.3">6.2.2.3</a>. Path Segment Normalization</h5> + + The complete path segments "." and ".." are intended only for use + within relative references (<a href="#section-4.1">Section 4.1</a>) and are removed as part of + the reference resolution process (<a href="#section-5.2">Section 5.2</a>). However, some + deployed implementations incorrectly assume that reference resolution + is not necessary when the reference is already a URI and thus fail to + remove dot-segments when they occur in non-relative paths. URI + normalizers should remove dot-segments by applying the + remove_dot_segments algorithm to the path, as described in + <a href="#section-5.2.4">Section 5.2.4</a>. + +<h4><a class="selflink" id="section-6.2.3" href="#section-6.2.3">6.2.3</a>. Scheme-Based Normalization</h4> + + The syntax and semantics of URIs vary from scheme to scheme, as + described by the defining specification for each scheme. + Implementations may use scheme-specific rules, at further processing + cost, to reduce the probability of false negatives. For example, + because the "http" scheme makes use of an authority component, has a + default port of "80", and defines an empty path to be equivalent to + "/", the following four URIs are equivalent: + + http://example.com + http://example.com/ + <a href="http://example.com:/">http://example.com:/</a> + <a href="http://example.com:80/">http://example.com:80/</a> + + In general, a URI that uses the generic syntax for authority with an + empty path should be normalized to a path of "/". Likewise, an + explicit ":port", for which the port is empty or the default for the + scheme, is equivalent to one where the port and its ":" delimiter are + elided and thus should be removed by scheme-based normalization. For + example, the second URI above is the normal form for the "http" + scheme. + + Another case where normalization varies by scheme is in the handling + of an empty authority component or empty host subcomponent. For many + scheme specifications, an empty authority or host is considered an + error; for others, it is considered equivalent to "localhost" or the + end-user's host. When a scheme defines a default for authority and a + URI reference to that default is desired, the reference should be + normalized to an empty authority for the sake of uniformity, brevity, + and internationalization. If, however, either the userinfo or port + subcomponents are non-empty, then the host should be given explicitly + even if it matches the default. + + Normalization should not remove delimiters when their associated + component is empty unless licensed to do so by the scheme + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 41]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-42"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + specification. For example, the URI "http://example.com/?" cannot be + assumed to be equivalent to any of the examples above. Likewise, the + presence or absence of delimiters within a userinfo subcomponent is + usually significant to its interpretation. The fragment component is + not subject to any scheme-based normalization; thus, two URIs that + differ only by the suffix "#" are considered different regardless of + the scheme. + + Some schemes define additional subcomponents that consist of case- + insensitive data, giving an implicit license to normalizers to + convert this data to a common case (e.g., all lowercase). For + example, URI schemes that define a subcomponent of path to contain an + Internet hostname, such as the "mailto" URI scheme, cause that + subcomponent to be case-insensitive and thus subject to case + normalization (e.g., "mailto:Joe@Example.COM" is equivalent to + "mailto:Joe@example.com", even though the generic syntax considers + the path component to be case-sensitive). + + Other scheme-specific normalizations are possible. + +<h4><a class="selflink" id="section-6.2.4" href="#section-6.2.4">6.2.4</a>. Protocol-Based Normalization</h4> + + Substantial effort to reduce the incidence of false negatives is + often cost-effective for web spiders. Therefore, they implement even + more aggressive techniques in URI comparison. For example, if they + observe that a URI such as + + http://example.com/data + + redirects to a URI differing only in the trailing slash + + http://example.com/data/ + + they will likely regard the two as equivalent in the future. This + kind of technique is only appropriate when equivalence is clearly + indicated by both the result of accessing the resources and the + common conventions of their scheme's dereference algorithm (in this + case, use of redirection by HTTP origin servers to avoid problems + with relative references). + + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 42]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-43"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h2><a class="selflink" id="section-7" href="#section-7">7</a>. Security Considerations</h2> + + A URI does not in itself pose a security threat. However, as URIs + are often used to provide a compact set of instructions for access to + network resources, care must be taken to properly interpret the data + within a URI, to prevent that data from causing unintended access, + and to avoid including data that should not be revealed in plain + text. + +<h3><a class="selflink" id="section-7.1" href="#section-7.1">7.1</a>. Reliability and Consistency</h3> + + There is no guarantee that once a URI has been used to retrieve + information, the same information will be retrievable by that URI in + the future. Nor is there any guarantee that the information + retrievable via that URI in the future will be observably similar to + that retrieved in the past. The URI syntax does not constrain how a + given scheme or authority apportions its namespace or maintains it + over time. Such guarantees can only be obtained from the person(s) + controlling that namespace and the resource in question. A specific + URI scheme may define additional semantics, such as name persistence, + if those semantics are required of all naming authorities for that + scheme. + +<h3><a class="selflink" id="section-7.2" href="#section-7.2">7.2</a>. Malicious Construction</h3> + + It is sometimes possible to construct a URI so that an attempt to + perform a seemingly harmless, idempotent operation, such as the + retrieval of a representation, will in fact cause a possibly damaging + remote operation. The unsafe URI is typically constructed by + specifying a port number other than that reserved for the network + protocol in question. The client unwittingly contacts a site running + a different protocol service, and data within the URI contains + instructions that, when interpreted according to this other protocol, + cause an unexpected operation. A frequent example of such abuse has + been the use of a protocol-based scheme with a port component of + "25", thereby fooling user agent software into sending an unintended + or impersonating message via an SMTP server. + + Applications should prevent dereference of a URI that specifies a TCP + port number within the "well-known port" range (0 - 1023) unless the + protocol being used to dereference that URI is compatible with the + protocol expected on that well-known port. Although IANA maintains a + registry of well-known ports, applications should make such + restrictions user-configurable to avoid preventing the deployment of + new services. + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 43]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-44"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + When a URI contains percent-encoded octets that match the delimiters + for a given resolution or dereference protocol (for example, CR and + LF characters for the TELNET protocol), these percent-encodings must + not be decoded before transmission across that protocol. Transfer of + the percent-encoding, which might violate the protocol, is less + harmful than allowing decoded octets to be interpreted as additional + operations or parameters, perhaps triggering an unexpected and + possibly harmful remote operation. + +<h3><a class="selflink" id="section-7.3" href="#section-7.3">7.3</a>. Back-End Transcoding</h3> + + When a URI is dereferenced, the data within it is often parsed by + both the user agent and one or more servers. In HTTP, for example, a + typical user agent will parse a URI into its five major components, + access the authority's server, and send it the data within the + authority, path, and query components. A typical server will take + that information, parse the path into segments and the query into + key/value pairs, and then invoke implementation-specific handlers to + respond to the request. As a result, a common security concern for + server implementations that handle a URI, either as a whole or split + into separate components, is proper interpretation of the octet data + represented by the characters and percent-encodings within that URI. + + Percent-encoded octets must be decoded at some point during the + dereference process. Applications must split the URI into its + components and subcomponents prior to decoding the octets, as + otherwise the decoded octets might be mistaken for delimiters. + Security checks of the data within a URI should be applied after + decoding the octets. Note, however, that the "%00" percent-encoding + (NUL) may require special handling and should be rejected if the + application is not expecting to receive raw data within a component. + + Special care should be taken when the URI path interpretation process + involves the use of a back-end file system or related system + functions. File systems typically assign an operational meaning to + special characters, such as the "/", "\", ":", "[", and "]" + characters, and to special device names like ".", "..", "...", "aux", + "lpt", etc. In some cases, merely testing for the existence of such + a name will cause the operating system to pause or invoke unrelated + system calls, leading to significant security concerns regarding + denial of service and unintended data transfer. It would be + impossible for this specification to list all such significant + characters and device names. Implementers should research the + reserved names and characters for the types of storage device that + may be attached to their applications and restrict the use of data + obtained from URI components accordingly. + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 44]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-45"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h3><a class="selflink" id="section-7.4" href="#section-7.4">7.4</a>. Rare IP Address Formats</h3> + + Although the URI syntax for IPv4address only allows the common + dotted-decimal form of IPv4 address literal, many implementations + that process URIs make use of platform-dependent system routines, + such as gethostbyname() and inet_aton(), to translate the string + literal to an actual IP address. Unfortunately, such system routines + often allow and process a much larger set of formats than those + described in <a href="#section-3.2.2">Section 3.2.2</a>. + + For example, many implementations allow dotted forms of three + numbers, wherein the last part is interpreted as a 16-bit quantity + and placed in the right-most two bytes of the network address (e.g., + a Class B network). Likewise, a dotted form of two numbers means + that the last part is interpreted as a 24-bit quantity and placed in + the right-most three bytes of the network address (Class A), and a + single number (without dots) is interpreted as a 32-bit quantity and + stored directly in the network address. Adding further to the + confusion, some implementations allow each dotted part to be + interpreted as decimal, octal, or hexadecimal, as specified in the C + language (i.e., a leading 0x or 0X implies hexadecimal; a leading 0 + implies octal; otherwise, the number is interpreted as decimal). + + These additional IP address formats are not allowed in the URI syntax + due to differences between platform implementations. However, they + can become a security concern if an application attempts to filter + access to resources based on the IP address in string literal format. + If this filtering is performed, literals should be converted to + numeric form and filtered based on the numeric value, and not on a + prefix or suffix of the string form. + +<h3><a class="selflink" id="section-7.5" href="#section-7.5">7.5</a>. Sensitive Information</h3> + + URI producers should not provide a URI that contains a username or + password that is intended to be secret. URIs are frequently + displayed by browsers, stored in clear text bookmarks, and logged by + user agent history and intermediary applications (proxies). A + password appearing within the userinfo component is deprecated and + should be considered an error (or simply ignored) except in those + rare cases where the 'password' parameter is intended to be public. + +<h3><a class="selflink" id="section-7.6" href="#section-7.6">7.6</a>. Semantic Attacks</h3> + + Because the userinfo subcomponent is rarely used and appears before + the host in the authority component, it can be used to construct a + URI intended to mislead a human user by appearing to identify one + (trusted) naming authority while actually identifying a different + authority hidden behind the noise. For example + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 45]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-46"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + ftp://cnn.example.com&story=breaking_news@10.0.0.1/top_story.htm + + might lead a human user to assume that the host is 'cnn.example.com', + whereas it is actually '10.0.0.1'. Note that a misleading userinfo + subcomponent could be much longer than the example above. + + A misleading URI, such as that above, is an attack on the user's + preconceived notions about the meaning of a URI rather than an attack + on the software itself. User agents may be able to reduce the impact + of such attacks by distinguishing the various components of the URI + when they are rendered, such as by using a different color or tone to + render userinfo if any is present, though there is no panacea. More + information on URI-based semantic attacks can be found in [<a href="#ref-Siedzik">Siedzik</a>]. + +<h2><a class="selflink" id="section-8" href="#section-8">8</a>. IANA Considerations</h2> + + URI scheme names, as defined by <scheme> in <a href="#section-3.1">Section 3.1</a>, form a + registered namespace that is managed by IANA according to the + procedures defined in [<a href="#ref-BCP35">BCP35</a>]. No IANA actions are required by this + document. + +<h2><a class="selflink" id="section-9" href="#section-9">9</a>. Acknowledgements</h2> + + This specification is derived from <a href="/doc/html/rfc2396">RFC 2396</a> [<a href="/doc/html/rfc2396">RFC2396</a>], <a href="/doc/html/rfc1808">RFC 1808</a> + [<a href="/doc/html/rfc1808">RFC1808</a>], and <a href="/doc/html/rfc1738">RFC 1738</a> [<a href="/doc/html/rfc1738">RFC1738</a>]; the acknowledgements in those + documents still apply. It also incorporates the update (with + corrections) for IPv6 literals in the host syntax, as defined by + Robert M. Hinden, Brian E. Carpenter, and Larry Masinter in + [<a href="/doc/html/rfc2732">RFC2732</a>]. In addition, contributions by Gisle Aas, Reese Anschultz, + Daniel Barclay, Tim Bray, Mike Brown, Rob Cameron, Jeremy Carroll, + Dan Connolly, Adam M. Costello, John Cowan, Jason Diamond, Martin + Duerst, Stefan Eissing, Clive D.W. Feather, Al Gilman, Tony Hammond, + Elliotte Harold, Pat Hayes, Henry Holtzman, Ian B. Jacobs, Michael + Kay, John C. Klensin, Graham Klyne, Dan Kohn, Bruce Lilly, Andrew + Main, Dave McAlpin, Ira McDonald, Michael Mealling, Ray Merkert, + Stephen Pollei, Julian Reschke, Tomas Rokicki, Miles Sabin, Kai + Schaetzl, Mark Thomson, Ronald Tschalaer, Norm Walsh, Marc Warne, + Stuart Williams, and Henry Zongaro are gratefully acknowledged. + +<h2><a class="selflink" id="section-10" href="#section-10">10</a>. References</h2> + +<h3><a class="selflink" id="section-10.1" href="#section-10.1">10.1</a>. Normative References</h3> + + [<a id="ref-ASCII">ASCII</a>] American National Standards Institute, "Coded Character + Set -- 7-bit American Standard Code for Information + Interchange", ANSI X3.4, 1986. + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 46]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-47"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + [<a id="ref-RFC2234">RFC2234</a>] Crocker, D. and P. Overell, "Augmented BNF for Syntax + Specifications: ABNF", <a href="/doc/html/rfc2234">RFC 2234</a>, November 1997. + + [<a id="ref-STD63">STD63</a>] Yergeau, F., "UTF-8, a transformation format of + ISO 10646", STD 63, <a href="/doc/html/rfc3629">RFC 3629</a>, November 2003. + + [<a id="ref-UCS">UCS</a>] International Organization for Standardization, + "Information Technology - Universal Multiple-Octet Coded + Character Set (UCS)", ISO/IEC 10646:2003, December 2003. + +<h3><a class="selflink" id="section-10.2" href="#section-10.2">10.2</a>. Informative References</h3> + + [<a id="ref-BCP19">BCP19</a>] Freed, N. and J. Postel, "IANA Charset Registration + Procedures", <a href="/doc/html/bcp19">BCP 19</a>, <a href="/doc/html/rfc2978">RFC 2978</a>, October 2000. + + [<a id="ref-BCP35">BCP35</a>] Petke, R. and I. King, "Registration Procedures for URL + Scheme Names", <a href="/doc/html/bcp35">BCP 35</a>, <a href="/doc/html/rfc2717">RFC 2717</a>, November 1999. + + [<a id="ref-RFC0952">RFC0952</a>] Harrenstien, K., Stahl, M., and E. Feinler, "DoD Internet + host table specification", <a href="/doc/html/rfc952">RFC 952</a>, October 1985. + + [<a id="ref-RFC1034">RFC1034</a>] Mockapetris, P., "Domain names - concepts and facilities", + STD 13, <a href="/doc/html/rfc1034">RFC 1034</a>, November 1987. + + [<a id="ref-RFC1123">RFC1123</a>] Braden, R., "Requirements for Internet Hosts - Application + and Support", STD 3, <a href="/doc/html/rfc1123">RFC 1123</a>, October 1989. + + [<a id="ref-RFC1535">RFC1535</a>] Gavron, E., "A Security Problem and Proposed Correction + With Widely Deployed DNS Software", <a href="/doc/html/rfc1535">RFC 1535</a>, + October 1993. + + [<a id="ref-RFC1630">RFC1630</a>] Berners-Lee, T., "Universal Resource Identifiers in WWW: A + Unifying Syntax for the Expression of Names and Addresses + of Objects on the Network as used in the World-Wide Web", + <a href="/doc/html/rfc1630">RFC 1630</a>, June 1994. + + [<a id="ref-RFC1736">RFC1736</a>] Kunze, J., "Functional Recommendations for Internet + Resource Locators", <a href="/doc/html/rfc1736">RFC 1736</a>, February 1995. + + [<a id="ref-RFC1737">RFC1737</a>] Sollins, K. and L. Masinter, "Functional Requirements for + Uniform Resource Names", <a href="/doc/html/rfc1737">RFC 1737</a>, December 1994. + + [<a id="ref-RFC1738">RFC1738</a>] Berners-Lee, T., Masinter, L., and M. McCahill, "Uniform + Resource Locators (URL)", <a href="/doc/html/rfc1738">RFC 1738</a>, December 1994. + + [<a id="ref-RFC1808">RFC1808</a>] Fielding, R., "Relative Uniform Resource Locators", + <a href="/doc/html/rfc1808">RFC 1808</a>, June 1995. + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 47]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-48"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + [<a id="ref-RFC2046">RFC2046</a>] Freed, N. and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part Two: Media Types", <a href="/doc/html/rfc2046">RFC 2046</a>, + November 1996. + + [<a id="ref-RFC2141">RFC2141</a>] Moats, R., "URN Syntax", <a href="/doc/html/rfc2141">RFC 2141</a>, May 1997. + + [<a id="ref-RFC2396">RFC2396</a>] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform + Resource Identifiers (URI): Generic Syntax", <a href="/doc/html/rfc2396">RFC 2396</a>, + August 1998. + + [<a id="ref-RFC2518">RFC2518</a>] Goland, Y., Whitehead, E., Faizi, A., Carter, S., and D. + Jensen, "HTTP Extensions for Distributed Authoring -- + WEBDAV", <a href="/doc/html/rfc2518">RFC 2518</a>, February 1999. + + [<a id="ref-RFC2557">RFC2557</a>] Palme, J., Hopmann, A., and N. Shelness, "MIME + Encapsulation of Aggregate Documents, such as HTML + (MHTML)", <a href="/doc/html/rfc2557">RFC 2557</a>, March 1999. + + [<a id="ref-RFC2718">RFC2718</a>] Masinter, L., Alvestrand, H., Zigmond, D., and R. Petke, + "Guidelines for new URL Schemes", <a href="/doc/html/rfc2718">RFC 2718</a>, November 1999. + + [<a id="ref-RFC2732">RFC2732</a>] Hinden, R., Carpenter, B., and L. Masinter, "Format for + Literal IPv6 Addresses in URL's", <a href="/doc/html/rfc2732">RFC 2732</a>, December 1999. + + [<a id="ref-RFC3305">RFC3305</a>] Mealling, M. and R. Denenberg, "Report from the Joint + W3C/IETF URI Planning Interest Group: Uniform Resource + Identifiers (URIs), URLs, and Uniform Resource Names + (URNs): Clarifications and Recommendations", <a href="/doc/html/rfc3305">RFC 3305</a>, + August 2002. + + [<a id="ref-RFC3490">RFC3490</a>] Faltstrom, P., Hoffman, P., and A. Costello, + "Internationalizing Domain Names in Applications (IDNA)", + <a href="/doc/html/rfc3490">RFC 3490</a>, March 2003. + + [<a id="ref-RFC3513">RFC3513</a>] Hinden, R. and S. Deering, "Internet Protocol Version 6 + (IPv6) Addressing Architecture", <a href="/doc/html/rfc3513">RFC 3513</a>, April 2003. + + [<a id="ref-Siedzik">Siedzik</a>] Siedzik, R., "Semantic Attacks: What's in a URL?", + April 2001, <<a href="https://www.giac.org/practical/gsec/Richard_Siedzik_GSEC.pdf">http://www.giac.org/practical/gsec/</a> + <a href="https://www.giac.org/practical/gsec/Richard_Siedzik_GSEC.pdf">Richard_Siedzik_GSEC.pdf</a>>. + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 48]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-49"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h2><a class="selflink" id="appendix-A" href="#appendix-A">Appendix A</a>. Collected ABNF for URI</h2> + + URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ] + + hier-part = "//" authority path-abempty + / path-absolute + / path-rootless + / path-empty + + URI-reference = URI / relative-ref + + absolute-URI = scheme ":" hier-part [ "?" query ] + + relative-ref = relative-part [ "?" query ] [ "#" fragment ] + + relative-part = "//" authority path-abempty + / path-absolute + / path-noscheme + / path-empty + + scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) + + authority = [ userinfo "@" ] host [ ":" port ] + userinfo = *( unreserved / pct-encoded / sub-delims / ":" ) + host = IP-literal / IPv4address / reg-name + port = *DIGIT + + IP-literal = "[" ( IPv6address / IPvFuture ) "]" + + IPvFuture = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" ) + + IPv6address = 6( h16 ":" ) ls32 + / "::" 5( h16 ":" ) ls32 + / [ h16 ] "::" 4( h16 ":" ) ls32 + / [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32 + / [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32 + / [ *3( h16 ":" ) h16 ] "::" h16 ":" ls32 + / [ *4( h16 ":" ) h16 ] "::" ls32 + / [ *5( h16 ":" ) h16 ] "::" h16 + / [ *6( h16 ":" ) h16 ] "::" + + h16 = 1*4HEXDIG + ls32 = ( h16 ":" h16 ) / IPv4address + IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 49]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-50"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + dec-octet = DIGIT ; 0-9 + / %x31-39 DIGIT ; 10-99 + / "1" 2DIGIT ; 100-199 + / "2" %x30-34 DIGIT ; 200-249 + / "25" %x30-35 ; 250-255 + + reg-name = *( unreserved / pct-encoded / sub-delims ) + + path = path-abempty ; begins with "/" or is empty + / path-absolute ; begins with "/" but not "//" + / path-noscheme ; begins with a non-colon segment + / path-rootless ; begins with a segment + / path-empty ; zero characters + + path-abempty = *( "/" segment ) + path-absolute = "/" [ segment-nz *( "/" segment ) ] + path-noscheme = segment-nz-nc *( "/" segment ) + path-rootless = segment-nz *( "/" segment ) + path-empty = 0<pchar> + + segment = *pchar + segment-nz = 1*pchar + segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" ) + ; non-zero-length segment without any colon ":" + + pchar = unreserved / pct-encoded / sub-delims / ":" / "@" + + query = *( pchar / "/" / "?" ) + + fragment = *( pchar / "/" / "?" ) + + pct-encoded = "%" HEXDIG HEXDIG + + unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" + reserved = gen-delims / sub-delims + gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@" + sub-delims = "!" / "$" / "&" / "'" / "(" / ")" + / "*" / "+" / "," / ";" / "=" + +<h2><a class="selflink" id="appendix-B" href="#appendix-B">Appendix B</a>. Parsing a URI Reference with a Regular Expression</h2> + + As the "first-match-wins" algorithm is identical to the "greedy" + disambiguation method used by POSIX regular expressions, it is + natural and commonplace to use a regular expression for parsing the + potential five components of a URI reference. + + The following line is the regular expression for breaking-down a + well-formed URI reference into its components. + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 50]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-51"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? + 12 3 4 5 6 7 8 9 + + The numbers in the second line above are only to assist readability; + they indicate the reference points for each subexpression (i.e., each + paired parenthesis). We refer to the value matched for subexpression + <n> as $<n>. For example, matching the above expression to + + <a href="http://www.ics.uci.edu/pub/ietf/uri/#Related">http://www.ics.uci.edu/pub/ietf/uri/#Related</a> + + results in the following subexpression matches: + + $1 = http: + $2 = http + $3 = //www.ics.uci.edu + $4 = www.ics.uci.edu + $5 = /pub/ietf/uri/ + $6 = <undefined> + $7 = <undefined> + $8 = #Related + $9 = Related + + where <undefined> indicates that the component is not present, as is + the case for the query component in the above example. Therefore, we + can determine the value of the five components as + + scheme = $2 + authority = $4 + path = $5 + query = $7 + fragment = $9 + + Going in the opposite direction, we can recreate a URI reference from + its components by using the algorithm of <a href="#section-5.3">Section 5.3</a>. + +<h2><a class="selflink" id="appendix-C" href="#appendix-C">Appendix C</a>. Delimiting a URI in Context</h2> + + URIs are often transmitted through formats that do not provide a + clear context for their interpretation. For example, there are many + occasions when a URI is included in plain text; examples include text + sent in email, USENET news, and on printed paper. In such cases, it + is important to be able to delimit the URI from the rest of the text, + and in particular from punctuation marks that might be mistaken for + part of the URI. + + In practice, URIs are delimited in a variety of ways, but usually + within double-quotes "http://example.com/", angle brackets + <http://example.com/>, or just by using whitespace: + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 51]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-52"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + http://example.com/ + + These wrappers do not form part of the URI. + + In some cases, extra whitespace (spaces, line-breaks, tabs, etc.) may + have to be added to break a long URI across lines. The whitespace + should be ignored when the URI is extracted. + + No whitespace should be introduced after a hyphen ("-") character. + Because some typesetters and printers may (erroneously) introduce a + hyphen at the end of line when breaking it, the interpreter of a URI + containing a line break immediately after a hyphen should ignore all + whitespace around the line break and should be aware that the hyphen + may or may not actually be part of the URI. + + Using <> angle brackets around each URI is especially recommended as + a delimiting style for a reference that contains embedded whitespace. + + The prefix "URL:" (with or without a trailing space) was formerly + recommended as a way to help distinguish a URI from other bracketed + designators, though it is not commonly used in practice and is no + longer recommended. + + For robustness, software that accepts user-typed URI should attempt + to recognize and strip both delimiters and embedded whitespace. + + For example, the text + + Yes, Jim, I found it under "http://www.w3.org/Addressing/", + but you can probably pick it up from <ftp://foo.example. + com/rfc/>. Note the warning in <<a href="http://www.ics.uci.edu/pub/ietf/uri/historical.html#WARNING">http://www.ics.uci.edu/pub/</a> + <a href="http://www.ics.uci.edu/pub/ietf/uri/historical.html#WARNING">ietf/uri/historical.html#WARNING</a>>. + + contains the URI references + + <a href="http://www.w3.org/Addressing/">http://www.w3.org/Addressing/</a> + ftp://foo.example.com/rfc/ + <a href="http://www.ics.uci.edu/pub/ietf/uri/historical.html#WARNING">http://www.ics.uci.edu/pub/ietf/uri/historical.html#WARNING</a> + + + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 52]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-53"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +<h2><a class="selflink" id="appendix-D" href="#appendix-D">Appendix D</a>. Changes from <a href="/doc/html/rfc2396">RFC 2396</a></h2> + +<h3><a class="selflink" id="appendix-D.1" href="#appendix-D.1">D.1</a>. Additions</h3> + + An ABNF rule for URI has been introduced to correspond to one common + usage of the term: an absolute URI with optional fragment. + + IPv6 (and later) literals have been added to the list of possible + identifiers for the host portion of an authority component, as + described by [<a href="/doc/html/rfc2732">RFC2732</a>], with the addition of "[" and "]" to the + reserved set and a version flag to anticipate future versions of IP + literals. Square brackets are now specified as reserved within the + authority component and are not allowed outside their use as + delimiters for an IP literal within host. In order to make this + change without changing the technical definition of the path, query, + and fragment components, those rules were redefined to directly + specify the characters allowed. + + As [<a href="/doc/html/rfc2732">RFC2732</a>] defers to [<a href="/doc/html/rfc3513">RFC3513</a>] for definition of an IPv6 literal + address, which, unfortunately, lacks an ABNF description of + IPv6address, we created a new ABNF rule for IPv6address that matches + the text representations defined by <a href="/doc/html/rfc3513#section-2.2">Section 2.2 of [RFC3513]</a>. + Likewise, the definition of IPv4address has been improved in order to + limit each decimal octet to the range 0-255. + + <a href="#section-6">Section 6</a>, on URI normalization and comparison, has been completely + rewritten and extended by using input from Tim Bray and discussion + within the W3C Technical Architecture Group. + +<h3><a class="selflink" id="appendix-D.2" href="#appendix-D.2">D.2</a>. Modifications</h3> + + The ad-hoc BNF syntax of <a href="/doc/html/rfc2396">RFC 2396</a> has been replaced with the ABNF of + [<a href="/doc/html/rfc2234">RFC2234</a>]. This change required all rule names that formerly + included underscore characters to be renamed with a dash instead. In + addition, a number of syntax rules have been eliminated or simplified + to make the overall grammar more comprehensible. Specifications that + refer to the obsolete grammar rules may be understood by replacing + those rules according to the following table: + + + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 53]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-54"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + +----------------+--------------------------------------------------+ + | obsolete rule | translation | + +----------------+--------------------------------------------------+ + | absoluteURI | absolute-URI | + | relativeURI | relative-part [ "?" query ] | + | hier_part | ( "//" authority path-abempty / | + | | path-absolute ) [ "?" query ] | + | | | + | opaque_part | path-rootless [ "?" query ] | + | net_path | "//" authority path-abempty | + | abs_path | path-absolute | + | rel_path | path-rootless | + | rel_segment | segment-nz-nc | + | reg_name | reg-name | + | server | authority | + | hostport | host [ ":" port ] | + | hostname | reg-name | + | path_segments | path-abempty | + | param | *<pchar excluding ";"> | + | | | + | uric | unreserved / pct-encoded / ";" / "?" / ":" | + | | / "@" / "&" / "=" / "+" / "$" / "," / "/" | + | | | + | uric_no_slash | unreserved / pct-encoded / ";" / "?" / ":" | + | | / "@" / "&" / "=" / "+" / "$" / "," | + | | | + | mark | "-" / "_" / "." / "!" / "~" / "*" / "'" | + | | / "(" / ")" | + | | | + | escaped | pct-encoded | + | hex | HEXDIG | + | alphanum | ALPHA / DIGIT | + +----------------+--------------------------------------------------+ + + Use of the above obsolete rules for the definition of scheme-specific + syntax is deprecated. + + <a href="#section-2">Section 2</a>, on characters, has been rewritten to explain what + characters are reserved, when they are reserved, and why they are + reserved, even when they are not used as delimiters by the generic + syntax. The mark characters that are typically unsafe to decode, + including the exclamation mark ("!"), asterisk ("*"), single-quote + ("'"), and open and close parentheses ("(" and ")"), have been moved + to the reserved set in order to clarify the distinction between + reserved and unreserved and, hopefully, to answer the most common + question of scheme designers. Likewise, the section on + percent-encoded characters has been rewritten, and URI normalizers + are now given license to decode any percent-encoded octets + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 54]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-55"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + corresponding to unreserved characters. In general, the terms + "escaped" and "unescaped" have been replaced with "percent-encoded" + and "decoded", respectively, to reduce confusion with other forms of + escape mechanisms. + + The ABNF for URI and URI-reference has been redesigned to make them + more friendly to LALR parsers and to reduce complexity. As a result, + the layout form of syntax description has been removed, along with + the uric, uric_no_slash, opaque_part, net_path, abs_path, rel_path, + path_segments, rel_segment, and mark rules. All references to + "opaque" URIs have been replaced with a better description of how the + path component may be opaque to hierarchy. The relativeURI rule has + been replaced with relative-ref to avoid unnecessary confusion over + whether they are a subset of URI. The ambiguity regarding the + parsing of URI-reference as a URI or a relative-ref with a colon in + the first segment has been eliminated through the use of five + separate path matching rules. + + The fragment identifier has been moved back into the section on + generic syntax components and within the URI and relative-ref rules, + though it remains excluded from absolute-URI. The number sign ("#") + character has been moved back to the reserved set as a result of + reintegrating the fragment syntax. + + The ABNF has been corrected to allow the path component to be empty. + This also allows an absolute-URI to consist of nothing after the + "scheme:", as is present in practice with the "dav:" namespace + [<a href="/doc/html/rfc2518">RFC2518</a>] and with the "about:" scheme used internally by many WWW + browser implementations. The ambiguity regarding the boundary + between authority and path has been eliminated through the use of + five separate path matching rules. + + Registry-based naming authorities that use the generic syntax are now + defined within the host rule. This change allows current + implementations, where whatever name provided is simply fed to the + local name resolution mechanism, to be consistent with the + specification. It also removes the need to re-specify DNS name + formats here. Furthermore, it allows the host component to contain + percent-encoded octets, which is necessary to enable + internationalized domain names to be provided in URIs, processed in + their native character encodings at the application layers above URI + processing, and passed to an IDNA library as a registered name in the + UTF-8 character encoding. The server, hostport, hostname, + domainlabel, toplabel, and alphanum rules have been removed. + + The resolving relative references algorithm of [<a href="/doc/html/rfc2396">RFC2396</a>] has been + rewritten with pseudocode for this revision to improve clarity and + fix the following issues: + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 55]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-56"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + o <a href="/doc/html/rfc2396#section-5.2">[RFC2396] section 5.2</a>, step 6a, failed to account for a base URI + with no path. + + o Restored the behavior of [<a href="/doc/html/rfc1808">RFC1808</a>] where, if the reference + contains an empty path and a defined query component, the target + URI inherits the base URI's path component. + + o The determination of whether a URI reference is a same-document + reference has been decoupled from the URI parser, simplifying the + URI processing interface within applications in a way consistent + with the internal architecture of deployed URI processing + implementations. The determination is now based on comparison to + the base URI after transforming a reference to absolute form, + rather than on the format of the reference itself. This change + may result in more references being considered "same-document" + under this specification than there would be under the rules given + in <a href="/doc/html/rfc2396">RFC 2396</a>, especially when normalization is used to reduce + aliases. However, it does not change the status of existing + same-document references. + + o Separated the path merge routine into two routines: merge, for + describing combination of the base URI path with a relative-path + reference, and remove_dot_segments, for describing how to remove + the special "." and ".." segments from a composed path. The + remove_dot_segments algorithm is now applied to all URI reference + paths in order to match common implementations and to improve the + normalization of URIs in practice. This change only impacts the + parsing of abnormal references and same-scheme references wherein + the base URI has a non-hierarchical path. + +Index + + A + ABNF 11 + absolute 27 + absolute-path 26 + absolute-URI 27 + access 9 + authority 17, 18 + + B + base URI 28 + + C + character encoding 4 + character 4 + characters 8, 11 + coded character set 4 + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 56]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-57"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + D + dec-octet 20 + dereference 9 + dot-segments 23 + + F + fragment 16, 24 + + G + gen-delims 13 + generic syntax 6 + + H + h16 20 + hier-part 16 + hierarchical 10 + host 18 + + I + identifier 5 + IP-literal 19 + IPv4 20 + IPv4address 19, 20 + IPv6 19 + IPv6address 19, 20 + IPvFuture 19 + + L + locator 7 + ls32 20 + + M + merge 32 + + N + name 7 + network-path 26 + + P + path 16, 22, 26 + path-abempty 22 + path-absolute 22 + path-empty 22 + path-noscheme 22 + path-rootless 22 + path-abempty 16, 22, 26 + path-absolute 16, 22, 26 + path-empty 16, 22, 26 + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 57]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-58"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + path-rootless 16, 22 + pchar 23 + pct-encoded 12 + percent-encoding 12 + port 22 + + Q + query 16, 23 + + R + reg-name 21 + registered name 20 + relative 10, 28 + relative-path 26 + relative-ref 26 + remove_dot_segments 33 + representation 9 + reserved 12 + resolution 9, 28 + resource 5 + retrieval 9 + + S + same-document 27 + sameness 9 + scheme 16, 17 + segment 22, 23 + segment-nz 23 + segment-nz-nc 23 + sub-delims 13 + suffix 27 + + T + transcription 8 + + U + uniform 4 + unreserved 13 + URI grammar + absolute-URI 27 + ALPHA 11 + authority 18 + CR 11 + dec-octet 20 + DIGIT 11 + DQUOTE 11 + fragment 24 + gen-delims 13 + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 58]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-59"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + + h16 20 + HEXDIG 11 + hier-part 16 + host 19 + IP-literal 19 + IPv4address 20 + IPv6address 20 + IPvFuture 19 + LF 11 + ls32 20 + OCTET 11 + path 22 + path-abempty 22 + path-absolute 22 + path-empty 22 + path-noscheme 22 + path-rootless 22 + pchar 23 + pct-encoded 12 + port 22 + query 24 + reg-name 21 + relative-ref 26 + reserved 13 + scheme 17 + segment 23 + segment-nz 23 + segment-nz-nc 23 + SP 11 + sub-delims 13 + unreserved 13 + URI 16 + URI-reference 25 + userinfo 18 + URI 16 + URI-reference 25 + URL 7 + URN 7 + userinfo 18 + + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 59]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-60"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +Authors' Addresses + + Tim Berners-Lee + World Wide Web Consortium + Massachusetts Institute of Technology + 77 Massachusetts Avenue + Cambridge, MA 02139 + USA + + Phone: +1-617-253-5702 + Fax: +1-617-258-5999 + EMail: timbl@w3.org + URI: <a href="http://www.w3.org/People/Berners-Lee/">http://www.w3.org/People/Berners-Lee/</a> + + + Roy T. Fielding + Day Software + 5251 California Ave., Suite 110 + Irvine, CA 92617 + USA + + Phone: +1-949-679-2960 + Fax: +1-949-679-2972 + EMail: fielding@gbiv.com + URI: <a href="http://roy.gbiv.com/">http://roy.gbiv.com/</a> + + + Larry Masinter + Adobe Systems Incorporated + 345 Park Ave + San Jose, CA 95110 + USA + + Phone: +1-408-536-3024 + EMail: LMM@acm.org + URI: <a href="http://larry.masinter.net/">http://larry.masinter.net/</a> + + + + + + + + + + + + + + + +<span class="grey">Berners-Lee, et al. Standards Track [Page 60]</span></pre> +<pre class="newpage"><hr class="noprint" id="page-61"><span class="grey"><a href="/doc/html/rfc3986">RFC 3986</a> URI Generic Syntax January 2005</span> + + +Full Copyright Statement + + Copyright (C) The Internet Society (2005). + + This document is subject to the rights, licenses and restrictions + contained in <a href="/doc/html/bcp78">BCP 78</a>, and except as set forth therein, the authors + retain all their rights. + + This document and the information contained herein are provided on an + "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS + OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET + ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, + INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE + INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED + WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Intellectual Property + + The IETF takes no position regarding the validity or scope of any + Intellectual Property Rights or other rights that might be claimed to + pertain to the implementation or use of the technology described in + this document or the extent to which any license under such rights + might or might not be available; nor does it represent that it has + made any independent effort to identify any such rights. Information + on the IETF's procedures with respect to rights in IETF Documents can + be found in <a href="/doc/html/bcp78">BCP 78</a> and <a href="/doc/html/bcp79">BCP 79</a>. + + Copies of IPR disclosures made to the IETF Secretariat and any + assurances of licenses to be made available, or the result of an + attempt made to obtain a general license or permission for the use of + such proprietary rights by implementers or users of this + specification can be obtained from the IETF on-line IPR repository at + <a href="http://www.ietf.org/ipr">http://www.ietf.org/ipr</a>. + + The IETF invites any interested party to bring to its attention any + copyrights, patents or patent applications, or other proprietary + rights that may cover technology that may be required to implement + this standard. Please address the information to the IETF at ietf- + ipr@ietf.org. + + +Acknowledgement + + Funding for the RFC Editor function is currently provided by the + Internet Society. + + + + + + +Berners-Lee, et al. Standards Track [Page 61] +</pre></div> + </div> + + </div> + <div class="d-print-none col-md-3 bg-light-subtle collapse show" id="sidebar"> + <div class="position-fixed border-start sidebar overflow-scroll overscroll-none no-scrollbar"> + <div class="d-flex flex-column vh-100 pt-2 pt-lg-3 ps-3 pl-md-2 pl-lg-3"> + <div> + <a class="btn btn-primary btn-sm" href="/doc/rfc3986/">Datatracker</a> + <p class="fw-bold pt-2"> + + RFC 3986 + + <br> + + + + + + +<span class="text-success">RFC + + - Internet Standard + +</span> + + </p> + </div> + + <ul class="nav nav-tabs nav-fill small me-2" role="tablist"> + <li class="nav-item" role="presentation" title="Document information"> + <button class="nav-link px-2" + id="docinfo-tab" + data-bs-toggle="tab" + data-bs-target="#docinfo-tab-pane" + type="button" + role="tab" + aria-controls="docinfo-tab-pane" + aria-selected="true"> + <i class="bi bi-info-circle"></i><span class="d-none d-md-block d-xl-inline ms-xl-1">Info</span> + </button> + </li> + <li class="nav-item" role="presentation" title="Table of contents"> + <button class="nav-link px-2" + id="toc-tab" + data-bs-toggle="tab" + data-bs-target="#toc-tab-pane" + type="button" + role="tab" + aria-controls="toc-tab-pane" + aria-selected="false"> + <i class="bi bi-list-ol"></i><span class="d-none d-md-block d-xl-inline ms-xl-1">Contents</span> + </button> + </li> + <li class="nav-item" role="presentation" title="Preferences"> + <button class="nav-link px-2" + id="pref-tab" + data-bs-toggle="tab" + data-bs-target="#pref-tab-pane" + type="button" + role="tab" + aria-controls="pref-tab-pane" + aria-selected="false"> + <i class="bi bi-gear"></i><span class="d-none d-md-block d-xl-inline ms-xl-1">Prefs</span> + </button> + </li> + </ul> + <div class="overflow-auto tab-content pt-2 me-2"> + <div class="tab-pane" + id="docinfo-tab-pane" + role="tabpanel" + aria-labelledby="docinfo-tab" + tabindex="0"> + <table class="table table-sm table-borderless"> + + + + + + + + +<tbody class="meta align-top "> + <tr> + <th scope="row">Document</th> + <th scope="row">Document type</th> + <td class="edit"></td> + <td> + + + + + + +<span class="text-success">RFC + + - Internet Standard + +</span> + + + + <br>January 2005 + + <br> + + <a class="btn btn-primary btn-sm my-1" + href="https://www.rfc-editor.org/errata_search.php?rfc=3986" title="Click to view errata." rel="nofollow"> + View errata + </a> + + + <a class="btn btn-sm btn-warning" + title="Click to report an error in the document." + href="https://www.rfc-editor.org/errata.php#reportnew" + target="_blank"> + Report errata + </a> + + + + <div>Updated by <a href="/doc/html/rfc8820" title="URI Design and Ownership">RFC 8820</a>, <a href="/doc/html/rfc7320" title="URI Design and Ownership">RFC 7320</a></div> + <div>Obsoletes <a href="/doc/html/rfc1808" title="Relative Uniform Resource Locators">RFC 1808</a>, <a href="/doc/html/rfc2732" title="Format for Literal IPv6 Addresses in URL's">RFC 2732</a>, <a href="/doc/html/rfc2396" title="Uniform Resource Identifiers (URI): Generic Syntax">RFC 2396</a></div> + <div>Updates <a href="/doc/html/rfc1738" title="Uniform Resource Locators (URL)">RFC 1738</a></div> + + + + <div> + Was + <a href="/doc/draft-fielding-uri-rfc2396bis/07/">draft-fielding-uri-rfc2396bis</a> + (individual in gen area) + </div> + + + + + + + + + + + + + + </td> + </tr> + + <tr> + <td></td> + <th scope="row">Select version</th> + <td class="edit"></td> + <td> + + + + + <ul class="revision-list pagination pagination-sm text-center flex-wrap my-0"> + + + + + <li class="page-item"> + <a class="page-link" + href="/doc/html/draft-fielding-uri-rfc2396bis-07" + rel="nofollow"> + 07 + </a> + </li> + + + + <li class="page-item rfc active"> + <a class="page-link" + href="/doc/html/rfc3986"> + RFC 3986 + </a> + </li> + + </ul> + + </td> + </tr> + + <tr> + <td></td> + <th scope="row">Compare versions</th> + <td class="edit"></td> + <td> + + + + +<form class="form-horizontal diff-form" + action="https://author-tools.ietf.org/iddiff" + method="get" + target="_blank"> + + <select class="form-select form-select-sm mb-1 select2-field" + data-max-entries="1" + data-width="resolve" + data-allow-clear="false" + data-minimum-input-length="0" + aria-label="From revision" + name="url1"> + + <option value="rfc3986"> + RFC 3986 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-07" selected> + draft-fielding-uri-rfc2396bis-07 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-06"> + draft-fielding-uri-rfc2396bis-06 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-05"> + draft-fielding-uri-rfc2396bis-05 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-04"> + draft-fielding-uri-rfc2396bis-04 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-03"> + draft-fielding-uri-rfc2396bis-03 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-02"> + draft-fielding-uri-rfc2396bis-02 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-01"> + draft-fielding-uri-rfc2396bis-01 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-00"> + draft-fielding-uri-rfc2396bis-00 + + </option> + + + </select> + + <select class="form-select form-select-sm mb-1 select2-field" + data-max-entries="1" + data-width="resolve" + data-allow-clear="false" + data-minimum-input-length="0" + aria-label="To revision" + name="url2"> + + <option value="rfc3986" selected> + RFC 3986 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-07"> + draft-fielding-uri-rfc2396bis-07 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-06"> + draft-fielding-uri-rfc2396bis-06 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-05"> + draft-fielding-uri-rfc2396bis-05 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-04"> + draft-fielding-uri-rfc2396bis-04 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-03"> + draft-fielding-uri-rfc2396bis-03 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-02"> + draft-fielding-uri-rfc2396bis-02 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-01"> + draft-fielding-uri-rfc2396bis-01 + + </option> + + <option value="draft-fielding-uri-rfc2396bis-00"> + draft-fielding-uri-rfc2396bis-00 + + </option> + + + </select> + + <button type="submit" + class="btn btn-primary btn-sm" + value="--html" + name="difftype"> + Side-by-side + </button> + + <button type="submit" + class="btn btn-primary btn-sm" + value="--hwdiff" + name="difftype"> + Inline + </button> + +</form> + </td> + </tr> + + + <tr> + <td></td> + <th scope="row">Authors</th> + <td class="edit"> + + </td> + <td> + + + <span ><a + title="Datatracker profile of Tim Berners-Lee" + href="/person/timbl@w3.org" >T. Berners-Lee</a> <a + href="mailto:timbl%40w3.org" + aria-label="Compose email to timbl@w3.org" + title="Compose email to timbl@w3.org"> + <i class="bi bi-envelope"></i></a></span>, + + <span ><a + title="Datatracker profile of Roy T. Fielding" + href="/person/fielding@gbiv.com" >R. Fielding</a> <a + href="mailto:fielding%40gbiv.com" + aria-label="Compose email to fielding@gbiv.com" + title="Compose email to fielding@gbiv.com"> + <i class="bi bi-envelope"></i></a></span>, + + <span ><a + title="Datatracker profile of Larry M Masinter" + href="/person/lmm@acm.org" >L. Masinter</a> <a + href="mailto:lmm%40acm.org" + aria-label="Compose email to lmm@acm.org" + title="Compose email to lmm@acm.org"> + <i class="bi bi-envelope"></i></a></span> + + + <br> + <a class="btn btn-primary btn-sm mt-1" href="mailto:rfc3986@ietf.org?subject=rfc3986" title="Send email to the document authors">Email authors</a> + + </td> + </tr> + + + <tr> + <td></td> + <th scope="row"> + RFC stream + </th> + <td class="edit"> + + </td> + <td > + + + + + + + + +<img alt="IETF Logo" + class="d-lm-none w-25 mt-1" + + + + src="https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor-white.svg" + + + > + +<img alt="IETF Logo" + class="d-dm-none w-25 mt-1" + + + + src="https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor.svg" + + + > + + + + + </td> + </tr> + + <tr> + <td></td> + <th scope="row"> + Other formats + </th> + <td class="edit"> + </td> + <td> + + + <div class="buttonlist"> + + + <a class="btn btn-primary btn-sm" + + target="_blank" + href="https://www.rfc-editor.org/rfc/rfc3986.txt"> + + <i class="bi bi-file-text"></i> txt + + </a> + + + + <a class="btn btn-primary btn-sm" + + target="_blank" + href="https://www.rfc-editor.org/rfc/rfc3986.html"> + + <i class="bi bi-file-code"></i> html + + </a> + + + + + + <a class="btn btn-primary btn-sm" + + target="_blank" + href="https://www.rfc-editor.org/rfc/inline-errata/rfc3986.html"> + + <i class="bi bi-file-diff"></i> w/errata + + </a> + + + + <a class="btn btn-primary btn-sm" + + target="_blank" + href="/doc/rfc3986/bibtex/"> + + <i class="bi bi-file-ruled"></i> bibtex + + </a> + + +</div> + + + </td> + </tr> + + + + +</tbody> + </table> + <a class="btn btn-sm btn-warning mb-3" + target="_blank" + href="https://github.com/ietf-tools/datatracker/issues/new/choose"> + Report a datatracker bug + <i class="bi bi-bug"></i> + </a> + </div> + <div class="tab-pane mb-5" + id="toc-tab-pane" + role="tabpanel" + aria-labelledby="toc-tab" + tabindex="0"> + <nav class="nav nav-pills flex-column small" id="toc-nav"> + </nav> + </div> + <div class="tab-pane mb-5 small" + id="pref-tab-pane" + role="tabpanel" + aria-labelledby="pref-tab" + tabindex="0"> + <label class="form-label fw-bold mb-2">Show sidebar by default</label> + <div class="btn-group-vertical btn-group-sm d-flex" role="group"> + <input type="radio" class="btn-check" name="sidebar" id="on-radio"> + <label class="btn btn-outline-primary" for="on-radio">Yes</label> + <input type="radio" class="btn-check" name="sidebar" id="off-radio"> + <label class="btn btn-outline-primary" for="off-radio">No</label> + </div> + <label class="form-label fw-bold mt-4 mb-2">Tab to show by default</label> + <div class="btn-group-vertical btn-group-sm d-flex" role="group"> + <input type="radio" class="btn-check" name="deftab" id="docinfo-radio"> + <label class="btn btn-outline-primary" for="docinfo-radio"> + <i class="bi bi-info-circle me-1"></i>Info + </label> + <input type="radio" class="btn-check" name="deftab" id="toc-radio"> + <label class="btn btn-outline-primary" for="toc-radio"> + <i class="bi bi-list-ol me-1"></i>Contents + </label> + </div> + <label class="form-label fw-bold mt-4 mb-2">HTMLization configuration</label> + <div class="btn-group-vertical btn-group-sm d-flex" role="group"> + <input type="radio" class="btn-check" name="htmlconf" id="txt-radio"> + <label class="btn btn-outline-primary" for="txt-radio" title="This is the traditional HTMLization method."> + <i class="bi bi-badge-sd me-1"></i>HTMLize the plaintext + </label> + <input type="radio" class="btn-check" name="htmlconf" id="html-radio"> + <label class="btn btn-outline-primary" for="html-radio" title="This is the modern HTMLization method."> + <i class="bi bi-badge-hd me-1"></i>Plaintextify the HTML + </label> + </div> + <label class="form-label fw-bold mt-4 mb-2" for="ptsize">Maximum font size</label> + <input type="range" class="form-range" min="7" max="16" id="ptsize" oninput="ptdemo.value = ptsize.value"> + <label class="form-label fw-bold mt-4 mb-2">Page dependencies</label> + <div class="btn-group-vertical btn-group-sm d-flex" role="group"> + <input type="radio" class="btn-check" name="pagedeps" id="inline-radio"> + <label class="btn btn-outline-primary" for="inline-radio" title="Generate larger, standalone web pages that do not require network access to render."> + <i class="bi bi-box me-1"></i>Inline + </label> + <input type="radio" class="btn-check" name="pagedeps" id="reference-radio"> + <label class="btn btn-outline-primary" for="reference-radio" title="Generate regular web pages that require network access to render."> + <i class="bi bi-link-45deg me-1"></i>Reference + </label> + </div> + <label class="form-label fw-bold mt-4 mb-2">Citation links</label> + <div class="btn-group-vertical btn-group-sm d-flex" role="group"> + <input type="radio" class="btn-check" name="reflinks" id="refsection-radio"> + <label class="btn btn-outline-primary" for="refsection-radio" title="Citation links go to the reference section."> + <i class="bi bi-arrow-clockwise"></i> Go to reference section + </label> + <input type="radio" class="btn-check" name="reflinks" id="citation-radio"> + <label class="btn btn-outline-primary" for="citation-radio" title="Citation links go directly to the cited document."> + <i class="bi bi-link-45deg me-1"></i>Go to linked document + </label> + </div> + </div> + </div> + </div> + </div> + </div> + </div> + +<script> + var _paq = window._paq || []; + + _paq.push(['disableCookies']); + _paq.push(['trackPageView']); + _paq.push(['enableLinkTracking']); + (function() { + var u="//analytics.ietf.org/"; + _paq.push(['setTrackerUrl', u+'matomo.php']); + _paq.push(['setSiteId', 7]); + var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; + g.type='text/javascript'; g.async=true; g.defer=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); + })(); +</script> +<noscript><p><img src="//analytics.ietf.org/matomo.php?idsite=7" style="border:0;" alt="" /></p></noscript> + + <script>(function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'a169414a2b79dd3f',t:'MTc4MzI4NDc2Mw=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();</script></body> +</html> diff --git a/test/current.md b/test/current.md @@ -0,0 +1,3351 @@ +Retrieved from [https://datatracker.ietf.org/doc/html/rfc3986](https://datatracker.ietf.org/doc/html/rfc3986) on Sun Jul 5 15:52:43 EST 2026. + +HTML. + +(IFRAME)// + -// Light// + -// Dark// + -// Auto// + + +////[ RFC 3986 + Internet Standard ](/doc/rfc3986/) Title | | Uniform Resource Identifier (URI): Generic Syntax + Document Document type | | RFC - Internet Standard +January 2005 +[ View errata ](https://www.rfc-editor.org/errata_search.php?rfc=3986)[ Report errata ](https://www.rfc-editor.org/errata.php#reportnew)Updated by [RFC 8820](/doc/html/rfc8820), [RFC 7320](/doc/html/rfc7320)Obsoletes [RFC 1808](/doc/html/rfc1808), [RFC 2732](/doc/html/rfc2732), [RFC 2396](/doc/html/rfc2396)Updates [RFC 1738](/doc/html/rfc1738) Was [draft-fielding-uri-rfc2396bis](/doc/draft-fielding-uri-rfc2396bis/07/) (individual in gen area) +| Select version | | + -[ 07 ](/doc/html/draft-fielding-uri-rfc2396bis-07) + -[ RFC 3986 ](/doc/html/rfc3986) + + + +| Compare versions | | +[FORM(No id) Side-by-side Inline ](get https://author-tools.ietf.org/iddiff) +| Authors | | [T. Berners-Lee](/person/timbl@w3.org)[//](mailto:timbl%40w3.org), [R. Fielding](/person/fielding@gbiv.com)[//](mailto:fielding%40gbiv.com), [L. Masinter](/person/lmm@acm.org)[//](mailto:lmm%40acm.org) +[Email authors](mailto:rfc3986@ietf.org?subject=rfc3986) +| RFC stream | | [img:IETF Logo](https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor-white.svg)[img:IETF Logo](https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor.svg) +| Other formats | | [// txt ](https://www.rfc-editor.org/rfc/rfc3986.txt)[// html ](https://www.rfc-editor.org/rfc/rfc3986.html)[// w/errata ](https://www.rfc-editor.org/rfc/inline-errata/rfc3986.html)[// bibtex ](/doc/rfc3986/bibtex/) + | | [ Report a bug //](https://github.com/ietf-tools/datatracker/issues/new/choose) + +`Network Working Group T. Berners-Lee +Request for Comments: 3986 W3C/MIT +STD: 66 R. Fielding +Updates: [1738](/doc/html/rfc1738) Day Software +Obsoletes: [2732](/doc/html/rfc2732), [2396](/doc/html/rfc2396), [1808](/doc/html/rfc1808) L. Masinter +Category: Standards Track Adobe Systems + January 2005 + + + +# Uniform Resource Identifier (URI): Generic Syntax + +Status of This Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (2005). + +Abstract + + A Uniform Resource Identifier (URI) is a compact sequence of + characters that identifies an abstract or physical resource. This + specification defines the generic URI syntax and a process for + resolving URI references that might be in relative form, along with + guidelines and security considerations for the use of URIs on the + Internet. The URI syntax defines a grammar that is a superset of all + valid URIs, allowing an implementation to parse the common components + of a URI reference without knowing the scheme-specific requirements + of every possible identifier. This specification does not define a + generative grammar for URIs; that task is performed by the individual + specifications of each URI scheme. + + + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 1]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + +Table of Contents + + [1](#section-1). Introduction . . . . . . . . . . . . . . . . . . . . . . . . . [4](#page-4)[1.1](#section-1.1). Overview of URIs . . . . . . . . . . . . . . . . . . . . [4](#page-4)[1.1.1](#section-1.1.1). Generic Syntax . . . . . . . . . . . . . . . . . [6](#page-6)[1.1.2](#section-1.1.2). Examples . . . . . . . . . . . . . . . . . . . . [7](#page-7)[1.1.3](#section-1.1.3). URI, URL, and URN . . . . . . . . . . . . . . . [7](#page-7)[1.2](#section-1.2). Design Considerations . . . . . . . . . . . . . . . . . [8](#page-8)[1.2.1](#section-1.2.1). Transcription . . . . . . . . . . . . . . . . . [8](#page-8)[1.2.2](#section-1.2.2). Separating Identification from Interaction . . . [9](#page-9)[1.2.3](#section-1.2.3). Hierarchical Identifiers . . . . . . . . . . . . [10](#page-10)[1.3](#section-1.3). Syntax Notation . . . . . . . . . . . . . . . . . . . . [11](#page-11)[2](#section-2). Characters . . . . . . . . . . . . . . . . . . . . . . . . . . [11](#page-11)[2.1](#section-2.1). Percent-Encoding . . . . . . . . . . . . . . . . . . . . [12](#page-12)[2.2](#section-2.2). Reserved Characters . . . . . . . . . . . . . . . . . . [12](#page-12)[2.3](#section-2.3). Unreserved Characters . . . . . . . . . . . . . . . . . [13](#page-13)[2.4](#section-2.4). When to Encode or Decode . . . . . . . . . . . . . . . . [14](#page-14)[2.5](#section-2.5). Identifying Data . . . . . . . . . . . . . . . . . . . . [14](#page-14)[3](#section-3). Syntax Components . . . . . . . . . . . . . . . . . . . . . . [16](#page-16)[3.1](#section-3.1). Scheme . . . . . . . . . . . . . . . . . . . . . . . . . [17](#page-17)[3.2](#section-3.2). Authority . . . . . . . . . . . . . . . . . . . . . . . [17](#page-17)[3.2.1](#section-3.2.1). User Information . . . . . . . . . . . . . . . . [18](#page-18)[3.2.2](#section-3.2.2). Host . . . . . . . . . . . . . . . . . . . . . . [18](#page-18)[3.2.3](#section-3.2.3). Port . . . . . . . . . . . . . . . . . . . . . . [22](#page-22)[3.3](#section-3.3). Path . . . . . . . . . . . . . . . . . . . . . . . . . . [22](#page-22)[3.4](#section-3.4). Query . . . . . . . . . . . . . . . . . . . . . . . . . [23](#page-23)[3.5](#section-3.5). Fragment . . . . . . . . . . . . . . . . . . . . . . . . [24](#page-24)[4](#section-4). Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . [25](#page-25)[4.1](#section-4.1). URI Reference . . . . . . . . . . . . . . . . . . . . . [25](#page-25)[4.2](#section-4.2). Relative Reference . . . . . . . . . . . . . . . . . . . [26](#page-26)[4.3](#section-4.3). Absolute URI . . . . . . . . . . . . . . . . . . . . . . [27](#page-27)[4.4](#section-4.4). Same-Document Reference . . . . . . . . . . . . . . . . [27](#page-27)[4.5](#section-4.5). Suffix Reference . . . . . . . . . . . . . . . . . . . . [27](#page-27)[5](#section-5). Reference Resolution . . . . . . . . . . . . . . . . . . . . . [28](#page-28)[5.1](#section-5.1). Establishing a Base URI . . . . . . . . . . . . . . . . [28](#page-28)[5.1.1](#section-5.1.1). Base URI Embedded in Content . . . . . . . . . . [29](#page-29)[5.1.2](#section-5.1.2). Base URI from the Encapsulating Entity . . . . . [29](#page-29)[5.1.3](#section-5.1.3). Base URI from the Retrieval URI . . . . . . . . [30](#page-30)[5.1.4](#section-5.1.4). Default Base URI . . . . . . . . . . . . . . . . [30](#page-30)[5.2](#section-5.2). Relative Resolution . . . . . . . . . . . . . . . . . . [30](#page-30)[5.2.1](#section-5.2.1). Pre-parse the Base URI . . . . . . . . . . . . . [31](#page-31)[5.2.2](#section-5.2.2). Transform References . . . . . . . . . . . . . . [31](#page-31)[5.2.3](#section-5.2.3). Merge Paths . . . . . . . . . . . . . . . . . . [32](#page-32)[5.2.4](#section-5.2.4). Remove Dot Segments . . . . . . . . . . . . . . [33](#page-33)[5.3](#section-5.3). Component Recomposition . . . . . . . . . . . . . . . . [35](#page-35)[5.4](#section-5.4). Reference Resolution Examples . . . . . . . . . . . . . [35](#page-35)[5.4.1](#section-5.4.1). Normal Examples . . . . . . . . . . . . . . . . [36](#page-36)[5.4.2](#section-5.4.2). Abnormal Examples . . . . . . . . . . . . . . . [36](#page-36)Berners-Lee, et al. Standards Track [Page 2]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005[6](#section-6). Normalization and Comparison . . . . . . . . . . . . . . . . . [38](#page-38)[6.1](#section-6.1). Equivalence . . . . . . . . . . . . . . . . . . . . . . [38](#page-38)[6.2](#section-6.2). Comparison Ladder . . . . . . . . . . . . . . . . . . . [39](#page-39)[6.2.1](#section-6.2.1). Simple String Comparison . . . . . . . . . . . . [39](#page-39)[6.2.2](#section-6.2.2). Syntax-Based Normalization . . . . . . . . . . . [40](#page-40)[6.2.3](#section-6.2.3). Scheme-Based Normalization . . . . . . . . . . . [41](#page-41)[6.2.4](#section-6.2.4). Protocol-Based Normalization . . . . . . . . . . [42](#page-42)[7](#section-7). Security Considerations . . . . . . . . . . . . . . . . . . . [43](#page-43)[7.1](#section-7.1). Reliability and Consistency . . . . . . . . . . . . . . [43](#page-43)[7.2](#section-7.2). Malicious Construction . . . . . . . . . . . . . . . . . [43](#page-43)[7.3](#section-7.3). Back-End Transcoding . . . . . . . . . . . . . . . . . . [44](#page-44)[7.4](#section-7.4). Rare IP Address Formats . . . . . . . . . . . . . . . . [45](#page-45)[7.5](#section-7.5). Sensitive Information . . . . . . . . . . . . . . . . . [45](#page-45)[7.6](#section-7.6). Semantic Attacks . . . . . . . . . . . . . . . . . . . . [45](#page-45)[8](#section-8). IANA Considerations . . . . . . . . . . . . . . . . . . . . . [46](#page-46)[9](#section-9). Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . [46](#page-46)[10](#section-10). References . . . . . . . . . . . . . . . . . . . . . . . . . . [46](#page-46)[10.1](#section-10.1). Normative References . . . . . . . . . . . . . . . . . . [46](#page-46)[10.2](#section-10.2). Informative References . . . . . . . . . . . . . . . . . [47](#page-47)[A](#appendix-A). Collected ABNF for URI . . . . . . . . . . . . . . . . . . . . [49](#page-49)[B](#appendix-B). Parsing a URI Reference with a Regular Expression . . . . . . [50](#page-50)[C](#appendix-C). Delimiting a URI in Context . . . . . . . . . . . . . . . . . [51](#page-51)[D](#appendix-D). Changes from [RFC 2396](/doc/html/rfc2396) . . . . . . . . . . . . . . . . . . . . [53](#page-53)[D.1](#appendix-D.1). Additions . . . . . . . . . . . . . . . . . . . . . . . [53](#page-53)[D.2](#appendix-D.2). Modifications . . . . . . . . . . . . . . . . . . . . . [53](#page-53) + Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . [56](#page-56) + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . [60](#page-60) + Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . [61](#page-61)Berners-Lee, et al. Standards Track [Page 3]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +## [1](#section-1). Introduction + + A Uniform Resource Identifier (URI) provides a simple and extensible + means for identifying a resource. This specification of URI syntax + and semantics is derived from concepts introduced by the World Wide + Web global information initiative, whose use of these identifiers + dates from 1990 and is described in "Universal Resource Identifiers + in WWW" [[RFC1630](/doc/html/rfc1630)]. The syntax is designed to meet the + recommendations laid out in "Functional Recommendations for Internet + Resource Locators" [[RFC1736](/doc/html/rfc1736)] and "Functional Requirements for Uniform + Resource Names" [[RFC1737](/doc/html/rfc1737)]. + + This document obsoletes [[RFC2396](/doc/html/rfc2396)], which merged "Uniform Resource + Locators" [[RFC1738](/doc/html/rfc1738)] and "Relative Uniform Resource Locators" + [[RFC1808](/doc/html/rfc1808)] in order to define a single, generic syntax for all URIs. + It obsoletes [[RFC2732](/doc/html/rfc2732)], which introduced syntax for an IPv6 address. + It excludes portions of [RFC 1738](/doc/html/rfc1738) that defined the specific syntax of + individual URI schemes; those portions will be updated as separate + documents. The process for registration of new URI schemes is + defined separately by [[BCP35](#ref-BCP35)]. Advice for designers of new URI + schemes can be found in [[RFC2718](/doc/html/rfc2718)]. All significant changes from [RFC](/doc/html/rfc2396)[2396](/doc/html/rfc2396) are noted in [Appendix D](#appendix-D). + + This specification uses the terms "character" and "coded character + set" in accordance with the definitions provided in [[BCP19](#ref-BCP19)], and + "character encoding" in place of what [[BCP19](#ref-BCP19)] refers to as a + "charset". + + +### [1.1](#section-1.1). Overview of URIs + + URIs are characterized as follows: + + Uniform + + Uniformity provides several benefits. It allows different types + of resource identifiers to be used in the same context, even when + the mechanisms used to access those resources may differ. It + allows uniform semantic interpretation of common syntactic + conventions across different types of resource identifiers. It + allows introduction of new types of resource identifiers without + interfering with the way that existing identifiers are used. It + allows the identifiers to be reused in many different contexts, + thus permitting new applications or protocols to leverage a pre- + existing, large, and widely used set of resource identifiers. + + + + + + + +Berners-Lee, et al. Standards Track [Page 4]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + Resource + + This specification does not limit the scope of what might be a + resource; rather, the term "resource" is used in a general sense + for whatever might be identified by a URI. Familiar examples + include an electronic document, an image, a source of information + with a consistent purpose (e.g., "today's weather report for Los + Angeles"), a service (e.g., an HTTP-to-SMS gateway), and a + collection of other resources. A resource is not necessarily + accessible via the Internet; e.g., human beings, corporations, and + bound books in a library can also be resources. Likewise, + abstract concepts can be resources, such as the operators and + operands of a mathematical equation, the types of a relationship + (e.g., "parent" or "employee"), or numeric values (e.g., zero, + one, and infinity). + + Identifier + + An identifier embodies the information required to distinguish + what is being identified from all other things within its scope of + identification. Our use of the terms "identify" and "identifying" + refer to this purpose of distinguishing one resource from all + other resources, regardless of how that purpose is accomplished + (e.g., by name, address, or context). These terms should not be + mistaken as an assumption that an identifier defines or embodies + the identity of what is referenced, though that may be the case + for some identifiers. Nor should it be assumed that a system + using URIs will access the resource identified: in many cases, + URIs are used to denote resources without any intention that they + be accessed. Likewise, the "one" resource identified might not be + singular in nature (e.g., a resource might be a named set or a + mapping that varies over time). + + A URI is an identifier consisting of a sequence of characters + matching the syntax rule named <URI> in [Section 3](#section-3). It enables + uniform identification of resources via a separately defined + extensible set of naming schemes ([Section 3.1](#section-3.1)). How that + identification is accomplished, assigned, or enabled is delegated to + each scheme specification. + + This specification does not place any limits on the nature of a + resource, the reasons why an application might seek to refer to a + resource, or the kinds of systems that might use URIs for the sake of + identifying resources. This specification does not require that a + URI persists in identifying the same resource over time, though that + is a common goal of all URI schemes. Nevertheless, nothing in this + + + + + +Berners-Lee, et al. Standards Track [Page 5]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + specification prevents an application from limiting itself to + particular types of resources, or to a subset of URIs that maintains + characteristics desired by that application. + + URIs have a global scope and are interpreted consistently regardless + of context, though the result of that interpretation may be in + relation to the end-user's context. For example, "http://localhost/" + has the same interpretation for every user of that reference, even + though the network interface corresponding to "localhost" may be + different for each end-user: interpretation is independent of access. + However, an action made on the basis of that reference will take + place in relation to the end-user's context, which implies that an + action intended to refer to a globally unique thing must use a URI + that distinguishes that resource from all other things. URIs that + identify in relation to the end-user's local context should only be + used when the context itself is a defining aspect of the resource, + such as when an on-line help manual refers to a file on the end- + user's file system (e.g., "file:///etc/hosts"). + + +#### [1.1.1](#section-1.1.1). Generic Syntax + + Each URI begins with a scheme name, as defined in [Section 3.1](#section-3.1), that + refers to a specification for assigning identifiers within that + scheme. As such, the URI syntax is a federated and extensible naming + system wherein each scheme's specification may further restrict the + syntax and semantics of identifiers using that scheme. + + This specification defines those elements of the URI syntax that are + required of all URI schemes or are common to many URI schemes. It + thus defines the syntax and semantics needed to implement a scheme- + independent parsing mechanism for URI references, by which the + scheme-dependent handling of a URI can be postponed until the + scheme-dependent semantics are needed. Likewise, protocols and data + formats that make use of URI references can refer to this + specification as a definition for the range of syntax allowed for all + URIs, including those schemes that have yet to be defined. This + decouples the evolution of identification schemes from the evolution + of protocols, data formats, and implementations that make use of + URIs. + + A parser of the generic URI syntax can parse any URI reference into + its major components. Once the scheme is determined, further + scheme-specific parsing can be performed on the components. In other + words, the URI generic syntax is a superset of the syntax of all URI + schemes. + + + + + + +Berners-Lee, et al. Standards Track [Page 6]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +#### [1.1.2](#section-1.1.2). Examples + + The following example URIs illustrate several URI schemes and + variations in their common syntax components: + + [ftp://ftp.is.co.za/rfc/rfc1808.txt](ftp://ftp.is.co.za/rfc/rfc1808.txt)[http://www.ietf.org/rfc/rfc2396.txt](http://www.ietf.org/rfc/rfc2396.txt) + + ldap://[2001:db8::7]/c=GB?objectClass?one + + mailto:John.Doe@example.com + + news:comp.infosystems.www.servers.unix + + tel:+1-816-555-1212 + + telnet://192.0.2.16:80/ + + urn:oasis:names:specification:docbook:dtd:xml:4.1.2 + + + +#### [1.1.3](#section-1.1.3). URI, URL, and URN + + A URI can be further classified as a locator, a name, or both. The + term "Uniform Resource Locator" (URL) refers to the subset of URIs + that, in addition to identifying a resource, provide a means of + locating the resource by describing its primary access mechanism + (e.g., its network "location"). The term "Uniform Resource Name" + (URN) has been used historically to refer to both URIs under the + "urn" scheme [[RFC2141](/doc/html/rfc2141)], which are required to remain globally unique + and persistent even when the resource ceases to exist or becomes + unavailable, and to any other URI with the properties of a name. + + An individual scheme does not have to be classified as being just one + of "name" or "locator". Instances of URIs from any given scheme may + have the characteristics of names or locators or both, often + depending on the persistence and care in the assignment of + identifiers by the naming authority, rather than on any quality of + the scheme. Future specifications and related documentation should + use the general term "URI" rather than the more restrictive terms + "URL" and "URN" [[RFC3305](/doc/html/rfc3305)]. + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 7]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +### [1.2](#section-1.2). Design Considerations +#### [1.2.1](#section-1.2.1). Transcription + + The URI syntax has been designed with global transcription as one of + its main considerations. A URI is a sequence of characters from a + very limited set: the letters of the basic Latin alphabet, digits, + and a few special characters. A URI may be represented in a variety + of ways; e.g., ink on paper, pixels on a screen, or a sequence of + character encoding octets. The interpretation of a URI depends only + on the characters used and not on how those characters are + represented in a network protocol. + + The goal of transcription can be described by a simple scenario. + Imagine two colleagues, Sam and Kim, sitting in a pub at an + international conference and exchanging research ideas. Sam asks Kim + for a location to get more information, so Kim writes the URI for the + research site on a napkin. Upon returning home, Sam takes out the + napkin and types the URI into a computer, which then retrieves the + information to which Kim referred. + + There are several design considerations revealed by the scenario: + + o A URI is a sequence of characters that is not always represented + as a sequence of octets. + + o A URI might be transcribed from a non-network source and thus + should consist of characters that are most likely able to be + entered into a computer, within the constraints imposed by + keyboards (and related input devices) across languages and + locales. + + o A URI often has to be remembered by people, and it is easier for + people to remember a URI when it consists of meaningful or + familiar components. + + These design considerations are not always in alignment. For + example, it is often the case that the most meaningful name for a URI + component would require characters that cannot be typed into some + systems. The ability to transcribe a resource identifier from one + medium to another has been considered more important than having a + URI consist of the most meaningful of components. + + In local or regional contexts and with improving technology, users + might benefit from being able to use a wider range of characters; + such use is not defined by this specification. Percent-encoded + octets ([Section 2.1](#section-2.1)) may be used within a URI to represent characters + outside the range of the US-ASCII coded character set if this + + + +Berners-Lee, et al. Standards Track [Page 8]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + representation is allowed by the scheme or by the protocol element in + which the URI is referenced. Such a definition should specify the + character encoding used to map those characters to octets prior to + being percent-encoded for the URI. + + +#### [1.2.2](#section-1.2.2). Separating Identification from Interaction + + A common misunderstanding of URIs is that they are only used to refer + to accessible resources. The URI itself only provides + identification; access to the resource is neither guaranteed nor + implied by the presence of a URI. Instead, any operation associated + with a URI reference is defined by the protocol element, data format + attribute, or natural language text in which it appears. + + Given a URI, a system may attempt to perform a variety of operations + on the resource, as might be characterized by words such as "access", + "update", "replace", or "find attributes". Such operations are + defined by the protocols that make use of URIs, not by this + specification. However, we do use a few general terms for describing + common operations on URIs. URI "resolution" is the process of + determining an access mechanism and the appropriate parameters + necessary to dereference a URI; this resolution may require several + iterations. To use that access mechanism to perform an action on the + URI's resource is to "dereference" the URI. + + When URIs are used within information retrieval systems to identify + sources of information, the most common form of URI dereference is + "retrieval": making use of a URI in order to retrieve a + representation of its associated resource. A "representation" is a + sequence of octets, along with representation metadata describing + those octets, that constitutes a record of the state of the resource + at the time when the representation is generated. Retrieval is + achieved by a process that might include using the URI as a cache key + to check for a locally cached representation, resolution of the URI + to determine an appropriate access mechanism (if any), and + dereference of the URI for the sake of applying a retrieval + operation. Depending on the protocols used to perform the retrieval, + additional information might be supplied about the resource (resource + metadata) and its relation to other resources. + + URI references in information retrieval systems are designed to be + late-binding: the result of an access is generally determined when it + is accessed and may vary over time or due to other aspects of the + interaction. These references are created in order to be used in the + future: what is being identified is not some specific result that was + obtained in the past, but rather some characteristic that is expected + to be true for future results. In such cases, the resource referred + to by the URI is actually a sameness of characteristics as observed + + + +Berners-Lee, et al. Standards Track [Page 9]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + over time, perhaps elucidated by additional comments or assertions + made by the resource provider. + + Although many URI schemes are named after protocols, this does not + imply that use of these URIs will result in access to the resource + via the named protocol. URIs are often used simply for the sake of + identification. Even when a URI is used to retrieve a representation + of a resource, that access might be through gateways, proxies, + caches, and name resolution services that are independent of the + protocol associated with the scheme name. The resolution of some + URIs may require the use of more than one protocol (e.g., both DNS + and HTTP are typically used to access an "http" URI's origin server + when a representation isn't found in a local cache). + + +#### [1.2.3](#section-1.2.3). Hierarchical Identifiers + + The URI syntax is organized hierarchically, with components listed in + order of decreasing significance from left to right. For some URI + schemes, the visible hierarchy is limited to the scheme itself: + everything after the scheme component delimiter (":") is considered + opaque to URI processing. Other URI schemes make the hierarchy + explicit and visible to generic parsing algorithms. + + The generic syntax uses the slash ("/"), question mark ("?"), and + number sign ("#") characters to delimit components that are + significant to the generic parser's hierarchical interpretation of an + identifier. In addition to aiding the readability of such + identifiers through the consistent use of familiar syntax, this + uniform representation of hierarchy across naming schemes allows + scheme-independent references to be made relative to that hierarchy. + + It is often the case that a group or "tree" of documents has been + constructed to serve a common purpose, wherein the vast majority of + URI references in these documents point to resources within the tree + rather than outside it. Similarly, documents located at a particular + site are much more likely to refer to other resources at that site + than to resources at remote sites. Relative referencing of URIs + allows document trees to be partially independent of their location + and access scheme. For instance, it is possible for a single set of + hypertext documents to be simultaneously accessible and traversable + via each of the "file", "http", and "ftp" schemes if the documents + refer to each other with relative references. Furthermore, such + document trees can be moved, as a whole, without changing any of the + relative references. + + A relative reference ([Section 4.2](#section-4.2)) refers to a resource by describing + the difference within a hierarchical name space between the reference + context and the target URI. The reference resolution algorithm, + + + +Berners-Lee, et al. Standards Track [Page 10]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + presented in [Section 5](#section-5), defines how such a reference is transformed + to the target URI. As relative references can only be used within + the context of a hierarchical URI, designers of new URI schemes + should use a syntax consistent with the generic syntax's hierarchical + components unless there are compelling reasons to forbid relative + referencing within that scheme. + + NOTE: Previous specifications used the terms "partial URI" and + "relative URI" to denote a relative reference to a URI. As some + readers misunderstood those terms to mean that relative URIs are a + subset of URIs rather than a method of referencing URIs, this + specification simply refers to them as relative references. + + All URI references are parsed by generic syntax parsers when used. + However, because hierarchical processing has no effect on an absolute + URI used in a reference unless it contains one or more dot-segments + (complete path segments of "." or "..", as described in [Section 3.3](#section-3.3)), + URI scheme specifications can define opaque identifiers by + disallowing use of slash characters, question mark characters, and + the URIs "scheme:." and "scheme:..". + + +### [1.3](#section-1.3). Syntax Notation + + This specification uses the Augmented Backus-Naur Form (ABNF) + notation of [[RFC2234](/doc/html/rfc2234)], including the following core ABNF syntax rules + defined by that specification: ALPHA (letters), CR (carriage return), + DIGIT (decimal digits), DQUOTE (double quote), HEXDIG (hexadecimal + digits), LF (line feed), and SP (space). The complete URI syntax is + collected in [Appendix A](#appendix-A). + + +## [2](#section-2). Characters + + The URI syntax provides a method of encoding data, presumably for the + sake of identifying a resource, as a sequence of characters. The URI + characters are, in turn, frequently encoded as octets for transport + or presentation. This specification does not mandate any particular + character encoding for mapping between URI characters and the octets + used to store or transmit those characters. When a URI appears in a + protocol element, the character encoding is defined by that protocol; + without such a definition, a URI is assumed to be in the same + character encoding as the surrounding text. + + The ABNF notation defines its terminal values to be non-negative + integers (codepoints) based on the US-ASCII coded character set + [[ASCII](#ref-ASCII)]. Because a URI is a sequence of characters, we must invert + that relation in order to understand the URI syntax. Therefore, the + + + + + +Berners-Lee, et al. Standards Track [Page 11]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + integer values used by the ABNF must be mapped back to their + corresponding characters via US-ASCII in order to complete the syntax + rules. + + A URI is composed from a limited set of characters consisting of + digits, letters, and a few graphic symbols. A reserved subset of + those characters may be used to delimit syntax components within a + URI while the remaining characters, including both the unreserved set + and those reserved characters not acting as delimiters, define each + component's identifying data. + + +### [2.1](#section-2.1). Percent-Encoding + + A percent-encoding mechanism is used to represent a data octet in a + component when that octet's corresponding character is outside the + allowed set or is being used as a delimiter of, or within, the + component. A percent-encoded octet is encoded as a character + triplet, consisting of the percent character "%" followed by the two + hexadecimal digits representing that octet's numeric value. For + example, "%20" is the percent-encoding for the binary octet + "00100000" (ABNF: %x20), which in US-ASCII corresponds to the space + character (SP). [Section 2.4](#section-2.4) describes when percent-encoding and + decoding is applied. + + pct-encoded = "%" HEXDIG HEXDIG + + The uppercase hexadecimal digits 'A' through 'F' are equivalent to + the lowercase digits 'a' through 'f', respectively. If two URIs + differ only in the case of hexadecimal digits used in percent-encoded + octets, they are equivalent. For consistency, URI producers and + normalizers should use uppercase hexadecimal digits for all percent- + encodings. + + +### [2.2](#section-2.2). Reserved Characters + + URIs include components and subcomponents that are delimited by + characters in the "reserved" set. These characters are called + "reserved" because they may (or may not) be defined as delimiters by + the generic syntax, by each scheme-specific syntax, or by the + implementation-specific syntax of a URI's dereferencing algorithm. + If data for a URI component would conflict with a reserved + character's purpose as a delimiter, then the conflicting data must be + percent-encoded before the URI is formed. + + + + + + + + +Berners-Lee, et al. Standards Track [Page 12]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + reserved = gen-delims / sub-delims + + gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@" + + sub-delims = "!" / "$" / "&" / "'" / "(" / ")" + / "*" / "+" / "," / ";" / "=" + + The purpose of reserved characters is to provide a set of delimiting + characters that are distinguishable from other data within a URI. + URIs that differ in the replacement of a reserved character with its + corresponding percent-encoded octet are not equivalent. Percent- + encoding a reserved character, or decoding a percent-encoded octet + that corresponds to a reserved character, will change how the URI is + interpreted by most applications. Thus, characters in the reserved + set are protected from normalization and are therefore safe to be + used by scheme-specific and producer-specific algorithms for + delimiting data subcomponents within a URI. + + A subset of the reserved characters (gen-delims) is used as + delimiters of the generic URI components described in [Section 3](#section-3). A + component's ABNF syntax rule will not use the reserved or gen-delims + rule names directly; instead, each syntax rule lists the characters + allowed within that component (i.e., not delimiting it), and any of + those characters that are also in the reserved set are "reserved" for + use as subcomponent delimiters within the component. Only the most + common subcomponents are defined by this specification; other + subcomponents may be defined by a URI scheme's specification, or by + the implementation-specific syntax of a URI's dereferencing + algorithm, provided that such subcomponents are delimited by + characters in the reserved set allowed within that component. + + URI producing applications should percent-encode data octets that + correspond to characters in the reserved set unless these characters + are specifically allowed by the URI scheme to represent data in that + component. If a reserved character is found in a URI component and + no delimiting role is known for that character, then it must be + interpreted as representing the data octet corresponding to that + character's encoding in US-ASCII. + + +### [2.3](#section-2.3). Unreserved Characters + + Characters that are allowed in a URI but do not have a reserved + purpose are called unreserved. These include uppercase and lowercase + letters, decimal digits, hyphen, period, underscore, and tilde. + + unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" + + + + + +Berners-Lee, et al. Standards Track [Page 13]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + URIs that differ in the replacement of an unreserved character with + its corresponding percent-encoded US-ASCII octet are equivalent: they + identify the same resource. However, URI comparison implementations + do not always perform normalization prior to comparison (see [Section](#section-6)[6](#section-6)). For consistency, percent-encoded octets in the ranges of ALPHA + (%41-%5A and %61-%7A), DIGIT (%30-%39), hyphen (%2D), period (%2E), + underscore (%5F), or tilde (%7E) should not be created by URI + producers and, when found in a URI, should be decoded to their + corresponding unreserved characters by URI normalizers. + + +### [2.4](#section-2.4). When to Encode or Decode + + Under normal circumstances, the only time when octets within a URI + are percent-encoded is during the process of producing the URI from + its component parts. This is when an implementation determines which + of the reserved characters are to be used as subcomponent delimiters + and which can be safely used as data. Once produced, a URI is always + in its percent-encoded form. + + When a URI is dereferenced, the components and subcomponents + significant to the scheme-specific dereferencing process (if any) + must be parsed and separated before the percent-encoded octets within + those components can be safely decoded, as otherwise the data may be + mistaken for component delimiters. The only exception is for + percent-encoded octets corresponding to characters in the unreserved + set, which can be decoded at any time. For example, the octet + corresponding to the tilde ("~") character is often encoded as "%7E" + by older URI processing implementations; the "%7E" can be replaced by + "~" without changing its interpretation. + + Because the percent ("%") character serves as the indicator for + percent-encoded octets, it must be percent-encoded as "%25" for that + octet to be used as data within a URI. Implementations must not + percent-encode or decode the same string more than once, as decoding + an already decoded string might lead to misinterpreting a percent + data octet as the beginning of a percent-encoding, or vice versa in + the case of percent-encoding an already percent-encoded string. + + +### [2.5](#section-2.5). Identifying Data + + URI characters provide identifying data for each of the URI + components, serving as an external interface for identification + between systems. Although the presence and nature of the URI + production interface is hidden from clients that use its URIs (and is + thus beyond the scope of the interoperability requirements defined by + this specification), it is a frequent source of confusion and errors + in the interpretation of URI character issues. Implementers have to + be aware that there are multiple character encodings involved in the + + + +Berners-Lee, et al. Standards Track [Page 14]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + production and transmission of URIs: local name and data encoding, + public interface encoding, URI character encoding, data format + encoding, and protocol encoding. + + Local names, such as file system names, are stored with a local + character encoding. URI producing applications (e.g., origin + servers) will typically use the local encoding as the basis for + producing meaningful names. The URI producer will transform the + local encoding to one that is suitable for a public interface and + then transform the public interface encoding into the restricted set + of URI characters (reserved, unreserved, and percent-encodings). + Those characters are, in turn, encoded as octets to be used as a + reference within a data format (e.g., a document charset), and such + data formats are often subsequently encoded for transmission over + Internet protocols. + + For most systems, an unreserved character appearing within a URI + component is interpreted as representing the data octet corresponding + to that character's encoding in US-ASCII. Consumers of URIs assume + that the letter "X" corresponds to the octet "01011000", and even + when that assumption is incorrect, there is no harm in making it. A + system that internally provides identifiers in the form of a + different character encoding, such as EBCDIC, will generally perform + character translation of textual identifiers to UTF-8 [[STD63](#ref-STD63)] (or + some other superset of the US-ASCII character encoding) at an + internal interface, thereby providing more meaningful identifiers + than those resulting from simply percent-encoding the original + octets. + + For example, consider an information service that provides data, + stored locally using an EBCDIC-based file system, to clients on the + Internet through an HTTP server. When an author creates a file with + the name "Laguna Beach" on that file system, the "http" URI + corresponding to that resource is expected to contain the meaningful + string "Laguna%20Beach". If, however, that server produces URIs by + using an overly simplistic raw octet mapping, then the result would + be a URI containing "%D3%81%87%A4%95%81@%C2%85%81%83%88". An + internal transcoding interface fixes this problem by transcoding the + local name to a superset of US-ASCII prior to producing the URI. + Naturally, proper interpretation of an incoming URI on such an + interface requires that percent-encoded octets be decoded (e.g., + "%20" to SP) before the reverse transcoding is applied to obtain the + local name. + + In some cases, the internal interface between a URI component and the + identifying data that it has been crafted to represent is much less + direct than a character encoding translation. For example, portions + of a URI might reflect a query on non-ASCII data, or numeric + + + +Berners-Lee, et al. Standards Track [Page 15]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + coordinates on a map. Likewise, a URI scheme may define components + with additional encoding requirements that are applied prior to + forming the component and producing the URI. + + When a new URI scheme defines a component that represents textual + data consisting of characters from the Universal Character Set [[UCS](#ref-UCS)], + the data should first be encoded as octets according to the UTF-8 + character encoding [[STD63](#ref-STD63)]; then only those octets that do not + correspond to characters in the unreserved set should be percent- + encoded. For example, the character A would be represented as "A", + the character LATIN CAPITAL LETTER A WITH GRAVE would be represented + as "%C3%80", and the character KATAKANA LETTER A would be represented + as "%E3%82%A2". + + +## [3](#section-3). Syntax Components + + The generic URI syntax consists of a hierarchical sequence of + components referred to as the scheme, authority, path, query, and + fragment. + + URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ] + + hier-part = "//" authority path-abempty + / path-absolute + / path-rootless + / path-empty + + The scheme and path components are required, though the path may be + empty (no characters). When authority is present, the path must + either be empty or begin with a slash ("/") character. When + authority is not present, the path cannot begin with two slash + characters ("//"). These restrictions result in five different ABNF + rules for a path ([Section 3.3](#section-3.3)), only one of which will match any + given URI reference. + + The following are two example URIs and their component parts: + + foo://example.com:8042/over/there?name=ferret#nose + \_/ \______________/\_________/ \_________/ \__/ + | | | | | + scheme authority path query fragment + | _____________________|__ + / \ / \ + urn:example:animal:ferret:nose + + + + + + + +Berners-Lee, et al. Standards Track [Page 16]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +### [3.1](#section-3.1). Scheme + + Each URI begins with a scheme name that refers to a specification for + assigning identifiers within that scheme. As such, the URI syntax is + a federated and extensible naming system wherein each scheme's + specification may further restrict the syntax and semantics of + identifiers using that scheme. + + Scheme names consist of a sequence of characters beginning with a + letter and followed by any combination of letters, digits, plus + ("+"), period ("."), or hyphen ("-"). Although schemes are case- + insensitive, the canonical form is lowercase and documents that + specify schemes must do so with lowercase letters. An implementation + should accept uppercase letters as equivalent to lowercase in scheme + names (e.g., allow "HTTP" as well as "http") for the sake of + robustness but should only produce lowercase scheme names for + consistency. + + scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) + + Individual schemes are not specified by this document. The process + for registration of new URI schemes is defined separately by [[BCP35](#ref-BCP35)]. + The scheme registry maintains the mapping between scheme names and + their specifications. Advice for designers of new URI schemes can be + found in [[RFC2718](/doc/html/rfc2718)]. URI scheme specifications must define their own + syntax so that all strings matching their scheme-specific syntax will + also match the <absolute-URI> grammar, as described in [Section 4.3](#section-4.3). + + When presented with a URI that violates one or more scheme-specific + restrictions, the scheme-specific resolution process should flag the + reference as an error rather than ignore the unused parts; doing so + reduces the number of equivalent URIs and helps detect abuses of the + generic syntax, which might indicate that the URI has been + constructed to mislead the user ([Section 7.6](#section-7.6)). + + +### [3.2](#section-3.2). Authority + + Many URI schemes include a hierarchical element for a naming + authority so that governance of the name space defined by the + remainder of the URI is delegated to that authority (which may, in + turn, delegate it further). The generic syntax provides a common + means for distinguishing an authority based on a registered name or + server address, along with optional port and user information. + + The authority component is preceded by a double slash ("//") and is + terminated by the next slash ("/"), question mark ("?"), or number + sign ("#") character, or by the end of the URI. + + + + +Berners-Lee, et al. Standards Track [Page 17]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + authority = [ userinfo "@" ] host [ ":" port ] + + URI producers and normalizers should omit the ":" delimiter that + separates host from port if the port component is empty. Some + schemes do not allow the userinfo and/or port subcomponents. + + If a URI contains an authority component, then the path component + must either be empty or begin with a slash ("/") character. Non- + validating parsers (those that merely separate a URI reference into + its major components) will often ignore the subcomponent structure of + authority, treating it as an opaque string from the double-slash to + the first terminating delimiter, until such time as the URI is + dereferenced. + + +#### [3.2.1](#section-3.2.1). User Information + + The userinfo subcomponent may consist of a user name and, optionally, + scheme-specific information about how to gain authorization to access + the resource. The user information, if present, is followed by a + commercial at-sign ("@") that delimits it from the host. + + userinfo = *( unreserved / pct-encoded / sub-delims / ":" ) + + Use of the format "user:password" in the userinfo field is + deprecated. Applications should not render as clear text any data + after the first colon (":") character found within a userinfo + subcomponent unless the data after the colon is the empty string + (indicating no password). Applications may choose to ignore or + reject such data when it is received as part of a reference and + should reject the storage of such data in unencrypted form. The + passing of authentication information in clear text has proven to be + a security risk in almost every case where it has been used. + + Applications that render a URI for the sake of user feedback, such as + in graphical hypertext browsing, should render userinfo in a way that + is distinguished from the rest of a URI, when feasible. Such + rendering will assist the user in cases where the userinfo has been + misleadingly crafted to look like a trusted domain name + ([Section 7.6](#section-7.6)). + + +#### [3.2.2](#section-3.2.2). Host + + The host subcomponent of authority is identified by an IP literal + encapsulated within square brackets, an IPv4 address in dotted- + decimal form, or a registered name. The host subcomponent is case- + insensitive. The presence of a host subcomponent within a URI does + not imply that the scheme requires access to the given host on the + Internet. In many cases, the host syntax is used only for the sake + + + +Berners-Lee, et al. Standards Track [Page 18]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + of reusing the existing registration process created and deployed for + DNS, thus obtaining a globally unique name without the cost of + deploying another registry. However, such use comes with its own + costs: domain name ownership may change over time for reasons not + anticipated by the URI producer. In other cases, the data within the + host component identifies a registered name that has nothing to do + with an Internet host. We use the name "host" for the ABNF rule + because that is its most common purpose, not its only purpose. + + host = IP-literal / IPv4address / reg-name + + The syntax rule for host is ambiguous because it does not completely + distinguish between an IPv4address and a reg-name. In order to + disambiguate the syntax, we apply the "first-match-wins" algorithm: + If host matches the rule for IPv4address, then it should be + considered an IPv4 address literal and not a reg-name. Although host + is case-insensitive, producers and normalizers should use lowercase + for registered names and hexadecimal addresses for the sake of + uniformity, while only using uppercase letters for percent-encodings. + + A host identified by an Internet Protocol literal address, version 6 + [[RFC3513](/doc/html/rfc3513)] or later, is distinguished by enclosing the IP literal + within square brackets ("[" and "]"). This is the only place where + square bracket characters are allowed in the URI syntax. In + anticipation of future, as-yet-undefined IP literal address formats, + an implementation may use an optional version flag to indicate such a + format explicitly rather than rely on heuristic determination. + + IP-literal = "[" ( IPv6address / IPvFuture ) "]" + + IPvFuture = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" ) + + The version flag does not indicate the IP version; rather, it + indicates future versions of the literal format. As such, + implementations must not provide the version flag for the existing + IPv4 and IPv6 literal address forms described below. If a URI + containing an IP-literal that starts with "v" (case-insensitive), + indicating that the version flag is present, is dereferenced by an + application that does not know the meaning of that version flag, then + the application should return an appropriate error for "address + mechanism not supported". + + A host identified by an IPv6 literal address is represented inside + the square brackets without a preceding version flag. The ABNF + provided here is a translation of the text definition of an IPv6 + literal address provided in [[RFC3513](/doc/html/rfc3513)]. This syntax does not support + IPv6 scoped addressing zone identifiers. + + + + +Berners-Lee, et al. Standards Track [Page 19]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + A 128-bit IPv6 address is divided into eight 16-bit pieces. Each + piece is represented numerically in case-insensitive hexadecimal, + using one to four hexadecimal digits (leading zeroes are permitted). + The eight encoded pieces are given most-significant first, separated + by colon characters. Optionally, the least-significant two pieces + may instead be represented in IPv4 address textual format. A + sequence of one or more consecutive zero-valued 16-bit pieces within + the address may be elided, omitting all their digits and leaving + exactly two consecutive colons in their place to mark the elision. + + IPv6address = 6( h16 ":" ) ls32 + / "::" 5( h16 ":" ) ls32 + / [ h16 ] "::" 4( h16 ":" ) ls32 + / [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32 + / [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32 + / [ *3( h16 ":" ) h16 ] "::" h16 ":" ls32 + / [ *4( h16 ":" ) h16 ] "::" ls32 + / [ *5( h16 ":" ) h16 ] "::" h16 + / [ *6( h16 ":" ) h16 ] "::" + + ls32 = ( h16 ":" h16 ) / IPv4address + ; least-significant 32 bits of address + + h16 = 1*4HEXDIG + ; 16 bits of address represented in hexadecimal + + A host identified by an IPv4 literal address is represented in + dotted-decimal notation (a sequence of four decimal numbers in the + range 0 to 255, separated by "."), as described in [[RFC1123](/doc/html/rfc1123)] by + reference to [[RFC0952](/doc/html/rfc0952)]. Note that other forms of dotted notation may + be interpreted on some platforms, as described in [Section 7.4](#section-7.4), but + only the dotted-decimal form of four octets is allowed by this + grammar. + + IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet + + dec-octet = DIGIT ; 0-9 + / %x31-39 DIGIT ; 10-99 + / "1" 2DIGIT ; 100-199 + / "2" %x30-34 DIGIT ; 200-249 + / "25" %x30-35 ; 250-255 + + A host identified by a registered name is a sequence of characters + usually intended for lookup within a locally defined host or service + name registry, though the URI's scheme-specific semantics may require + that a specific registry (or fixed name table) be used instead. The + most common name registry mechanism is the Domain Name System (DNS). + A registered name intended for lookup in the DNS uses the syntax + + + +Berners-Lee, et al. Standards Track [Page 20]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + defined in [Section 3.5 of [RFC1034]](/doc/html/rfc1034#section-3.5) and [Section 2.1 of [RFC1123]](/doc/html/rfc1123#section-2.1). + Such a name consists of a sequence of domain labels separated by ".", + each domain label starting and ending with an alphanumeric character + and possibly also containing "-" characters. The rightmost domain + label of a fully qualified domain name in DNS may be followed by a + single "." and should be if it is necessary to distinguish between + the complete domain name and some local domain. + + reg-name = *( unreserved / pct-encoded / sub-delims ) + + If the URI scheme defines a default for host, then that default + applies when the host subcomponent is undefined or when the + registered name is empty (zero length). For example, the "file" URI + scheme is defined so that no authority, an empty host, and + "localhost" all mean the end-user's machine, whereas the "http" + scheme considers a missing authority or empty host invalid. + + This specification does not mandate a particular registered name + lookup technology and therefore does not restrict the syntax of reg- + name beyond what is necessary for interoperability. Instead, it + delegates the issue of registered name syntax conformance to the + operating system of each application performing URI resolution, and + that operating system decides what it will allow for the purpose of + host identification. A URI resolution implementation might use DNS, + host tables, yellow pages, NetInfo, WINS, or any other system for + lookup of registered names. However, a globally scoped naming + system, such as DNS fully qualified domain names, is necessary for + URIs intended to have global scope. URI producers should use names + that conform to the DNS syntax, even when use of DNS is not + immediately apparent, and should limit these names to no more than + 255 characters in length. + + The reg-name syntax allows percent-encoded octets in order to + represent non-ASCII registered names in a uniform way that is + independent of the underlying name resolution technology. Non-ASCII + characters must first be encoded according to UTF-8 [[STD63](#ref-STD63)], and then + each octet of the corresponding UTF-8 sequence must be percent- + encoded to be represented as URI characters. URI producing + applications must not use percent-encoding in host unless it is used + to represent a UTF-8 character sequence. When a non-ASCII registered + name represents an internationalized domain name intended for + resolution via the DNS, the name must be transformed to the IDNA + encoding [[RFC3490](/doc/html/rfc3490)] prior to name lookup. URI producers should + provide these registered names in the IDNA encoding, rather than a + percent-encoding, if they wish to maximize interoperability with + legacy URI resolvers. + + + + + +Berners-Lee, et al. Standards Track [Page 21]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +#### [3.2.3](#section-3.2.3). Port + + The port subcomponent of authority is designated by an optional port + number in decimal following the host and delimited from it by a + single colon (":") character. + + port = *DIGIT + + A scheme may define a default port. For example, the "http" scheme + defines a default port of "80", corresponding to its reserved TCP + port number. The type of port designated by the port number (e.g., + TCP, UDP, SCTP) is defined by the URI scheme. URI producers and + normalizers should omit the port component and its ":" delimiter if + port is empty or if its value would be the same as that of the + scheme's default. + + +### [3.3](#section-3.3). Path + + The path component contains data, usually organized in hierarchical + form, that, along with data in the non-hierarchical query component + ([Section 3.4](#section-3.4)), serves to identify a resource within the scope of the + URI's scheme and naming authority (if any). The path is terminated + by the first question mark ("?") or number sign ("#") character, or + by the end of the URI. + + If a URI contains an authority component, then the path component + must either be empty or begin with a slash ("/") character. If a URI + does not contain an authority component, then the path cannot begin + with two slash characters ("//"). In addition, a URI reference + ([Section 4.1](#section-4.1)) may be a relative-path reference, in which case the + first path segment cannot contain a colon (":") character. The ABNF + requires five separate rules to disambiguate these cases, only one of + which will match the path substring within a given URI reference. We + use the generic term "path component" to describe the URI substring + matched by the parser to one of these rules. + + path = path-abempty ; begins with "/" or is empty + / path-absolute ; begins with "/" but not "//" + / path-noscheme ; begins with a non-colon segment + / path-rootless ; begins with a segment + / path-empty ; zero characters + + path-abempty = *( "/" segment ) + path-absolute = "/" [ segment-nz *( "/" segment ) ] + path-noscheme = segment-nz-nc *( "/" segment ) + path-rootless = segment-nz *( "/" segment ) + path-empty = 0<pchar> + + + + +Berners-Lee, et al. Standards Track [Page 22]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + segment = *pchar + segment-nz = 1*pchar + segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" ) + ; non-zero-length segment without any colon ":" + + pchar = unreserved / pct-encoded / sub-delims / ":" / "@" + + A path consists of a sequence of path segments separated by a slash + ("/") character. A path is always defined for a URI, though the + defined path may be empty (zero length). Use of the slash character + to indicate hierarchy is only required when a URI will be used as the + context for relative references. For example, the URI + <mailto:fred@example.com> has a path of "fred@example.com", whereas + the URI <foo://info.example.com?fred> has an empty path. + + The path segments "." and "..", also known as dot-segments, are + defined for relative reference within the path name hierarchy. They + are intended for use at the beginning of a relative-path reference + ([Section 4.2](#section-4.2)) to indicate relative position within the hierarchical + tree of names. This is similar to their role within some operating + systems' file directory structures to indicate the current directory + and parent directory, respectively. However, unlike in a file + system, these dot-segments are only interpreted within the URI path + hierarchy and are removed as part of the resolution process ([Section](#section-5.2)[5.2](#section-5.2)). + + Aside from dot-segments in hierarchical paths, a path segment is + considered opaque by the generic syntax. URI producing applications + often use the reserved characters allowed in a segment to delimit + scheme-specific or dereference-handler-specific subcomponents. For + example, the semicolon (";") and equals ("=") reserved characters are + often used to delimit parameters and parameter values applicable to + that segment. The comma (",") reserved character is often used for + similar purposes. For example, one URI producer might use a segment + such as "name;v=1.1" to indicate a reference to version 1.1 of + "name", whereas another might use a segment such as "name,1.1" to + indicate the same. Parameter types may be defined by scheme-specific + semantics, but in most cases the syntax of a parameter is specific to + the implementation of the URI's dereferencing algorithm. + + +### [3.4](#section-3.4). Query + + The query component contains non-hierarchical data that, along with + data in the path component ([Section 3.3](#section-3.3)), serves to identify a + resource within the scope of the URI's scheme and naming authority + (if any). The query component is indicated by the first question + mark ("?") character and terminated by a number sign ("#") character + or by the end of the URI. + + + +Berners-Lee, et al. Standards Track [Page 23]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + query = *( pchar / "/" / "?" ) + + The characters slash ("/") and question mark ("?") may represent data + within the query component. Beware that some older, erroneous + implementations may not handle such data correctly when it is used as + the base URI for relative references ([Section 5.1](#section-5.1)), apparently + because they fail to distinguish query data from path data when + looking for hierarchical separators. However, as query components + are often used to carry identifying information in the form of + "key=value" pairs and one frequently used value is a reference to + another URI, it is sometimes better for usability to avoid percent- + encoding those characters. + + +### [3.5](#section-3.5). Fragment + + The fragment identifier component of a URI allows indirect + identification of a secondary resource by reference to a primary + resource and additional identifying information. The identified + secondary resource may be some portion or subset of the primary + resource, some view on representations of the primary resource, or + some other resource defined or described by those representations. A + fragment identifier component is indicated by the presence of a + number sign ("#") character and terminated by the end of the URI. + + fragment = *( pchar / "/" / "?" ) + + The semantics of a fragment identifier are defined by the set of + representations that might result from a retrieval action on the + primary resource. The fragment's format and resolution is therefore + dependent on the media type [[RFC2046](/doc/html/rfc2046)] of a potentially retrieved + representation, even though such a retrieval is only performed if the + URI is dereferenced. If no such representation exists, then the + semantics of the fragment are considered unknown and are effectively + unconstrained. Fragment identifier semantics are independent of the + URI scheme and thus cannot be redefined by scheme specifications. + + Individual media types may define their own restrictions on or + structures within the fragment identifier syntax for specifying + different types of subsets, views, or external references that are + identifiable as secondary resources by that media type. If the + primary resource has multiple representations, as is often the case + for resources whose representation is selected based on attributes of + the retrieval request (a.k.a., content negotiation), then whatever is + identified by the fragment should be consistent across all of those + representations. Each representation should either define the + fragment so that it corresponds to the same secondary resource, + regardless of how it is represented, or should leave the fragment + undefined (i.e., not found). + + + +Berners-Lee, et al. Standards Track [Page 24]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + As with any URI, use of a fragment identifier component does not + imply that a retrieval action will take place. A URI with a fragment + identifier may be used to refer to the secondary resource without any + implication that the primary resource is accessible or will ever be + accessed. + + Fragment identifiers have a special role in information retrieval + systems as the primary form of client-side indirect referencing, + allowing an author to specifically identify aspects of an existing + resource that are only indirectly provided by the resource owner. As + such, the fragment identifier is not used in the scheme-specific + processing of a URI; instead, the fragment identifier is separated + from the rest of the URI prior to a dereference, and thus the + identifying information within the fragment itself is dereferenced + solely by the user agent, regardless of the URI scheme. Although + this separate handling is often perceived to be a loss of + information, particularly for accurate redirection of references as + resources move over time, it also serves to prevent information + providers from denying reference authors the right to refer to + information within a resource selectively. Indirect referencing also + provides additional flexibility and extensibility to systems that use + URIs, as new media types are easier to define and deploy than new + schemes of identification. + + The characters slash ("/") and question mark ("?") are allowed to + represent data within the fragment identifier. Beware that some + older, erroneous implementations may not handle this data correctly + when it is used as the base URI for relative references ([Section](#section-5.1)[5.1](#section-5.1)). + + +## [4](#section-4). Usage + + When applications make reference to a URI, they do not always use the + full form of reference defined by the "URI" syntax rule. To save + space and take advantage of hierarchical locality, many Internet + protocol elements and media type formats allow an abbreviation of a + URI, whereas others restrict the syntax to a particular form of URI. + We define the most common forms of reference syntax in this + specification because they impact and depend upon the design of the + generic syntax, requiring a uniform parsing algorithm in order to be + interpreted consistently. + + +### [4.1](#section-4.1). URI Reference + + URI-reference is used to denote the most common usage of a resource + identifier. + + URI-reference = URI / relative-ref + + + +Berners-Lee, et al. Standards Track [Page 25]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + A URI-reference is either a URI or a relative reference. If the + URI-reference's prefix does not match the syntax of a scheme followed + by its colon separator, then the URI-reference is a relative + reference. + + A URI-reference is typically parsed first into the five URI + components, in order to determine what components are present and + whether the reference is relative. Then, each component is parsed + for its subparts and their validation. The ABNF of URI-reference, + along with the "first-match-wins" disambiguation rule, is sufficient + to define a validating parser for the generic syntax. Readers + familiar with regular expressions should see [Appendix B](#appendix-B) for an + example of a non-validating URI-reference parser that will take any + given string and extract the URI components. + + +### [4.2](#section-4.2). Relative Reference + + A relative reference takes advantage of the hierarchical syntax + ([Section 1.2.3](#section-1.2.3)) to express a URI reference relative to the name space + of another hierarchical URI. + + relative-ref = relative-part [ "?" query ] [ "#" fragment ] + + relative-part = "//" authority path-abempty + / path-absolute + / path-noscheme + / path-empty + + The URI referred to by a relative reference, also known as the target + URI, is obtained by applying the reference resolution algorithm of + [Section 5](#section-5). + + A relative reference that begins with two slash characters is termed + a network-path reference; such references are rarely used. A + relative reference that begins with a single slash character is + termed an absolute-path reference. A relative reference that does + not begin with a slash character is termed a relative-path reference. + + A path segment that contains a colon character (e.g., "this:that") + cannot be used as the first segment of a relative-path reference, as + it would be mistaken for a scheme name. Such a segment must be + preceded by a dot-segment (e.g., "./this:that") to make a relative- + path reference. + + + + + + + + +Berners-Lee, et al. Standards Track [Page 26]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +### [4.3](#section-4.3). Absolute URI + + Some protocol elements allow only the absolute form of a URI without + a fragment identifier. For example, defining a base URI for later + use by relative references calls for an absolute-URI syntax rule that + does not allow a fragment. + + absolute-URI = scheme ":" hier-part [ "?" query ] + + URI scheme specifications must define their own syntax so that all + strings matching their scheme-specific syntax will also match the + <absolute-URI> grammar. Scheme specifications will not define + fragment identifier syntax or usage, regardless of its applicability + to resources identifiable via that scheme, as fragment identification + is orthogonal to scheme definition. However, scheme specifications + are encouraged to include a wide range of examples, including + examples that show use of the scheme's URIs with fragment identifiers + when such usage is appropriate. + + +### [4.4](#section-4.4). Same-Document Reference + + When a URI reference refers to a URI that is, aside from its fragment + component (if any), identical to the base URI ([Section 5.1](#section-5.1)), that + reference is called a "same-document" reference. The most frequent + examples of same-document references are relative references that are + empty or include only the number sign ("#") separator followed by a + fragment identifier. + + When a same-document reference is dereferenced for a retrieval + action, the target of that reference is defined to be within the same + entity (representation, document, or message) as the reference; + therefore, a dereference should not result in a new retrieval action. + + Normalization of the base and target URIs prior to their comparison, + as described in Sections [6.2.2](#section-6.2.2) and [6.2.3](#section-6.2.3), is allowed but rarely + performed in practice. Normalization may increase the set of same- + document references, which may be of benefit to some caching + applications. As such, reference authors should not assume that a + slightly different, though equivalent, reference URI will (or will + not) be interpreted as a same-document reference by any given + application. + + +### [4.5](#section-4.5). Suffix Reference + + The URI syntax is designed for unambiguous reference to resources and + extensibility via the URI scheme. However, as URI identification and + usage have become commonplace, traditional media (television, radio, + newspapers, billboards, etc.) have increasingly used a suffix of the + + + +Berners-Lee, et al. Standards Track [Page 27]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + URI as a reference, consisting of only the authority and path + portions of the URI, such as + + www.w3.org/Addressing/ + + or simply a DNS registered name on its own. Such references are + primarily intended for human interpretation rather than for machines, + with the assumption that context-based heuristics are sufficient to + complete the URI (e.g., most registered names beginning with "www" + are likely to have a URI prefix of "http://"). Although there is no + standard set of heuristics for disambiguating a URI suffix, many + client implementations allow them to be entered by the user and + heuristically resolved. + + Although this practice of using suffix references is common, it + should be avoided whenever possible and should never be used in + situations where long-term references are expected. The heuristics + noted above will change over time, particularly when a new URI scheme + becomes popular, and are often incorrect when used out of context. + Furthermore, they can lead to security issues along the lines of + those described in [[RFC1535](/doc/html/rfc1535)]. + + As a URI suffix has the same syntax as a relative-path reference, a + suffix reference cannot be used in contexts where a relative + reference is expected. As a result, suffix references are limited to + places where there is no defined base URI, such as dialog boxes and + off-line advertisements. + + +## [5](#section-5). Reference Resolution + + This section defines the process of resolving a URI reference within + a context that allows relative references so that the result is a + string matching the <URI> syntax rule of [Section 3](#section-3). + + +### [5.1](#section-5.1). Establishing a Base URI + + The term "relative" implies that a "base URI" exists against which + the relative reference is applied. Aside from fragment-only + references ([Section 4.4](#section-4.4)), relative references are only usable when a + base URI is known. A base URI must be established by the parser + prior to parsing URI references that might be relative. A base URI + must conform to the <absolute-URI> syntax rule ([Section 4.3](#section-4.3)). If the + base URI is obtained from a URI reference, then that reference must + be converted to absolute form and stripped of any fragment component + prior to its use as a base URI. + + + + + + +Berners-Lee, et al. Standards Track [Page 28]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + The base URI of a reference can be established in one of four ways, + discussed below in order of precedence. The order of precedence can + be thought of in terms of layers, where the innermost defined base + URI has the highest precedence. This can be visualized graphically + as follows: + + .----------------------------------------------------------. + | .----------------------------------------------------. | + | | .----------------------------------------------. | | + | | | .----------------------------------------. | | | + | | | | .----------------------------------. | | | | + | | | | | <relative-reference> | | | | | + | | | | `----------------------------------' | | | | + | | | | (5.1.1) Base URI embedded in content | | | | + | | | `----------------------------------------' | | | + | | | (5.1.2) Base URI of the encapsulating entity | | | + | | | (message, representation, or none) | | | + | | `----------------------------------------------' | | + | | (5.1.3) URI used to retrieve the entity | | + | `----------------------------------------------------' | + | (5.1.4) Default Base URI (application-dependent) | + `----------------------------------------------------------' + + +#### [5.1.1](#section-5.1.1). Base URI Embedded in Content + + Within certain media types, a base URI for relative references can be + embedded within the content itself so that it can be readily obtained + by a parser. This can be useful for descriptive documents, such as + tables of contents, which may be transmitted to others through + protocols other than their usual retrieval context (e.g., email or + USENET news). + + It is beyond the scope of this specification to specify how, for each + media type, a base URI can be embedded. The appropriate syntax, when + available, is described by the data format specification associated + with each media type. + + +#### [5.1.2](#section-5.1.2). Base URI from the Encapsulating Entity + + If no base URI is embedded, the base URI is defined by the + representation's retrieval context. For a document that is enclosed + within another entity, such as a message or archive, the retrieval + context is that entity. Thus, the default base URI of a + representation is the base URI of the entity in which the + representation is encapsulated. + + + + + + +Berners-Lee, et al. Standards Track [Page 29]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + A mechanism for embedding a base URI within MIME container types + (e.g., the message and multipart types) is defined by MHTML + [[RFC2557](/doc/html/rfc2557)]. Protocols that do not use the MIME message header syntax, + but that do allow some form of tagged metadata to be included within + messages, may define their own syntax for defining a base URI as part + of a message. + + +#### [5.1.3](#section-5.1.3). Base URI from the Retrieval URI + + If no base URI is embedded and the representation is not encapsulated + within some other entity, then, if a URI was used to retrieve the + representation, that URI shall be considered the base URI. Note that + if the retrieval was the result of a redirected request, the last URI + used (i.e., the URI that resulted in the actual retrieval of the + representation) is the base URI. + + +#### [5.1.4](#section-5.1.4). Default Base URI + + If none of the conditions described above apply, then the base URI is + defined by the context of the application. As this definition is + necessarily application-dependent, failing to define a base URI by + using one of the other methods may result in the same content being + interpreted differently by different types of applications. + + A sender of a representation containing relative references is + responsible for ensuring that a base URI for those references can be + established. Aside from fragment-only references, relative + references can only be used reliably in situations where the base URI + is well defined. + + +### [5.2](#section-5.2). Relative Resolution + + This section describes an algorithm for converting a URI reference + that might be relative to a given base URI into the parsed components + of the reference's target. The components can then be recomposed, as + described in [Section 5.3](#section-5.3), to form the target URI. This algorithm + provides definitive results that can be used to test the output of + other implementations. Applications may implement relative reference + resolution by using some other algorithm, provided that the results + match what would be given by this one. + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 30]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +#### [5.2.1](#section-5.2.1). Pre-parse the Base URI + + The base URI (Base) is established according to the procedure of + [Section 5.1](#section-5.1) and parsed into the five main components described in + [Section 3](#section-3). Note that only the scheme component is required to be + present in a base URI; the other components may be empty or + undefined. A component is undefined if its associated delimiter does + not appear in the URI reference; the path component is never + undefined, though it may be empty. + + Normalization of the base URI, as described in Sections [6.2.2](#section-6.2.2) and + 6.2.3, is optional. A URI reference must be transformed to its + target URI before it can be normalized. + + +#### [5.2.2](#section-5.2.2). Transform References + + For each URI reference (R), the following pseudocode describes an + algorithm for transforming R into its target URI (T): + + -- The URI reference is parsed into the five URI components + -- + (R.scheme, R.authority, R.path, R.query, R.fragment) = parse(R); + + -- A non-strict parser may ignore a scheme in the reference + -- if it is identical to the base URI's scheme. + -- + if ((not strict) and (R.scheme == Base.scheme)) then + undefine(R.scheme); + endif; + + + + + + + + + + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 31]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + if defined(R.scheme) then + T.scheme = R.scheme; + T.authority = R.authority; + T.path = remove_dot_segments(R.path); + T.query = R.query; + else + if defined(R.authority) then + T.authority = R.authority; + T.path = remove_dot_segments(R.path); + T.query = R.query; + else + if (R.path == "") then + T.path = Base.path; + if defined(R.query) then + T.query = R.query; + else + T.query = Base.query; + endif; + else + if (R.path starts-with "/") then + T.path = remove_dot_segments(R.path); + else + T.path = merge(Base.path, R.path); + T.path = remove_dot_segments(T.path); + endif; + T.query = R.query; + endif; + T.authority = Base.authority; + endif; + T.scheme = Base.scheme; + endif; + + T.fragment = R.fragment; + + +#### [5.2.3](#section-5.2.3). Merge Paths + + The pseudocode above refers to a "merge" routine for merging a + relative-path reference with the path of the base URI. This is + accomplished as follows: + + o If the base URI has a defined authority component and an empty + path, then return a string consisting of "/" concatenated with the + reference's path; otherwise, + + + + + + + + +Berners-Lee, et al. Standards Track [Page 32]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + o return a string consisting of the reference's path component + appended to all but the last segment of the base URI's path (i.e., + excluding any characters after the right-most "/" in the base URI + path, or excluding the entire base URI path if it does not contain + any "/" characters). + + +#### [5.2.4](#section-5.2.4). Remove Dot Segments + + The pseudocode also refers to a "remove_dot_segments" routine for + interpreting and removing the special "." and ".." complete path + segments from a referenced path. This is done after the path is + extracted from a reference, whether or not the path was relative, in + order to remove any invalid or extraneous dot-segments prior to + forming the target URI. Although there are many ways to accomplish + this removal process, we describe a simple method using two string + buffers. + + 1. The input buffer is initialized with the now-appended path + components and the output buffer is initialized to the empty + string. + + 2. While the input buffer is not empty, loop as follows: + + A. If the input buffer begins with a prefix of "../" or "./", + then remove that prefix from the input buffer; otherwise, + + B. if the input buffer begins with a prefix of "/./" or "/.", + where "." is a complete path segment, then replace that + prefix with "/" in the input buffer; otherwise, + + C. if the input buffer begins with a prefix of "/../" or "/..", + where ".." is a complete path segment, then replace that + prefix with "/" in the input buffer and remove the last + segment and its preceding "/" (if any) from the output + buffer; otherwise, + + D. if the input buffer consists only of "." or "..", then remove + that from the input buffer; otherwise, + + E. move the first path segment in the input buffer to the end of + the output buffer, including the initial "/" character (if + any) and any subsequent characters up to, but not including, + the next "/" character or the end of the input buffer. + + 3. Finally, the output buffer is returned as the result of + remove_dot_segments. + + + + + +Berners-Lee, et al. Standards Track [Page 33]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + Note that dot-segments are intended for use in URI references to + express an identifier relative to the hierarchy of names in the base + URI. The remove_dot_segments algorithm respects that hierarchy by + removing extra dot-segments rather than treat them as an error or + leaving them to be misinterpreted by dereference implementations. + + The following illustrates how the above steps are applied for two + examples of merged paths, showing the state of the two buffers after + each step. + + STEP OUTPUT BUFFER INPUT BUFFER + + 1 : /a/b/c/./../../g + 2E: /a /b/c/./../../g + 2E: /a/b /c/./../../g + 2E: /a/b/c /./../../g + 2B: /a/b/c /../../g + 2C: /a/b /../g + 2C: /a /g + 2E: /a/g + + STEP OUTPUT BUFFER INPUT BUFFER + + [1](#section-1) : mid/content=5/../6 + 2E: mid /content=5/../6 + 2E: mid/content=5 /../6 + 2C: mid /6 + 2E: mid/6 + + Some applications may find it more efficient to implement the + remove_dot_segments algorithm by using two segment stacks rather than + strings. + + Note: Beware that some older, erroneous implementations will fail + to separate a reference's query component from its path component + prior to merging the base and reference paths, resulting in an + interoperability failure if the query component contains the + strings "/../" or "/./". + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 34]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +### [5.3](#section-5.3). Component Recomposition + + Parsed URI components can be recomposed to obtain the corresponding + URI reference string. Using pseudocode, this would be: + + result = "" + + if defined(scheme) then + append scheme to result; + append ":" to result; + endif; + + if defined(authority) then + append "//" to result; + append authority to result; + endif; + + append path to result; + + if defined(query) then + append "?" to result; + append query to result; + endif; + + if defined(fragment) then + append "#" to result; + append fragment to result; + endif; + + return result; + + Note that we are careful to preserve the distinction between a + component that is undefined, meaning that its separator was not + present in the reference, and a component that is empty, meaning that + the separator was present and was immediately followed by the next + component separator or the end of the reference. + + +### [5.4](#section-5.4). Reference Resolution Examples + + Within a representation with a well defined base URI of + + http://a/b/c/d;p?q + + a relative reference is transformed to its target URI as follows. + + + + + + + +Berners-Lee, et al. Standards Track [Page 35]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +#### [5.4.1](#section-5.4.1). Normal Examples + + "g:h" = "g:h" + "g" = "http://a/b/c/g" + "./g" = "http://a/b/c/g" + "g/" = "http://a/b/c/g/" + "/g" = "http://a/g" + "//g" = "http://g" + "?y" = "http://a/b/c/d;p?y" + "g?y" = "http://a/b/c/g?y" + "#s" = "http://a/b/c/d;p?q#s" + "g#s" = "http://a/b/c/g#s" + "g?y#s" = "http://a/b/c/g?y#s" + ";x" = "http://a/b/c/;x" + "g;x" = "http://a/b/c/g;x" + "g;x?y#s" = "http://a/b/c/g;x?y#s" + "" = "http://a/b/c/d;p?q" + "." = "http://a/b/c/" + "./" = "http://a/b/c/" + ".." = "http://a/b/" + "../" = "http://a/b/" + "../g" = "http://a/b/g" + "../.." = "http://a/" + "../../" = "http://a/" + "../../g" = "http://a/g" + + +#### [5.4.2](#section-5.4.2). Abnormal Examples + + Although the following abnormal examples are unlikely to occur in + normal practice, all URI parsers should be capable of resolving them + consistently. Each example uses the same base as that above. + + Parsers must be careful in handling cases where there are more ".." + segments in a relative-path reference than there are hierarchical + levels in the base URI's path. Note that the ".." syntax cannot be + used to change the authority component of a URI. + + "../../../g" = "http://a/g" + "../../../../g" = "http://a/g" + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 36]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + Similarly, parsers must remove the dot-segments "." and ".." when + they are complete components of a path, but not when they are only + part of a segment. + + "/./g" = "http://a/g" + "/../g" = "http://a/g" + "g." = "http://a/b/c/g." + ".g" = "http://a/b/c/.g" + "g.." = "http://a/b/c/g.." + "..g" = "http://a/b/c/..g" + + Less likely are cases where the relative reference uses unnecessary + or nonsensical forms of the "." and ".." complete path segments. + + "./../g" = "http://a/b/g" + "./g/." = "http://a/b/c/g/" + "g/./h" = "http://a/b/c/g/h" + "g/../h" = "http://a/b/c/h" + "g;x=1/./y" = "http://a/b/c/g;x=1/y" + "g;x=1/../y" = "http://a/b/c/y" + + Some applications fail to separate the reference's query and/or + fragment components from the path component before merging it with + the base path and removing dot-segments. This error is rarely + noticed, as typical usage of a fragment never includes the hierarchy + ("/") character and the query component is not normally used within + relative references. + + "g?y/./x" = "http://a/b/c/g?y/./x" + "g?y/../x" = "http://a/b/c/g?y/../x" + "g#s/./x" = "http://a/b/c/g#s/./x" + "g#s/../x" = "http://a/b/c/g#s/../x" + + Some parsers allow the scheme name to be present in a relative + reference if it is the same as the base URI scheme. This is + considered to be a loophole in prior specifications of partial URI + [[RFC1630](/doc/html/rfc1630)]. Its use should be avoided but is allowed for backward + compatibility. + + "http:g" = "http:g" ; for strict parsers + / "http://a/b/c/g" ; for backward compatibility + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 37]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +## [6](#section-6). Normalization and Comparison + + One of the most common operations on URIs is simple comparison: + determining whether two URIs are equivalent without using the URIs to + access their respective resource(s). A comparison is performed every + time a response cache is accessed, a browser checks its history to + color a link, or an XML parser processes tags within a namespace. + Extensive normalization prior to comparison of URIs is often used by + spiders and indexing engines to prune a search space or to reduce + duplication of request actions and response storage. + + URI comparison is performed for some particular purpose. Protocols + or implementations that compare URIs for different purposes will + often be subject to differing design trade-offs in regards to how + much effort should be spent in reducing aliased identifiers. This + section describes various methods that may be used to compare URIs, + the trade-offs between them, and the types of applications that might + use them. + + +### [6.1](#section-6.1). Equivalence + + Because URIs exist to identify resources, presumably they should be + considered equivalent when they identify the same resource. However, + this definition of equivalence is not of much practical use, as there + is no way for an implementation to compare two resources unless it + has full knowledge or control of them. For this reason, + determination of equivalence or difference of URIs is based on string + comparison, perhaps augmented by reference to additional rules + provided by URI scheme definitions. We use the terms "different" and + "equivalent" to describe the possible outcomes of such comparisons, + but there are many application-dependent versions of equivalence. + + Even though it is possible to determine that two URIs are equivalent, + URI comparison is not sufficient to determine whether two URIs + identify different resources. For example, an owner of two different + domain names could decide to serve the same resource from both, + resulting in two different URIs. Therefore, comparison methods are + designed to minimize false negatives while strictly avoiding false + positives. + + In testing for equivalence, applications should not directly compare + relative references; the references should be converted to their + respective target URIs before comparison. When URIs are compared to + select (or avoid) a network action, such as retrieval of a + representation, fragment components (if any) should be excluded from + the comparison. + + + + + +Berners-Lee, et al. Standards Track [Page 38]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +### [6.2](#section-6.2). Comparison Ladder + + A variety of methods are used in practice to test URI equivalence. + These methods fall into a range, distinguished by the amount of + processing required and the degree to which the probability of false + negatives is reduced. As noted above, false negatives cannot be + eliminated. In practice, their probability can be reduced, but this + reduction requires more processing and is not cost-effective for all + applications. + + If this range of comparison practices is considered as a ladder, the + following discussion will climb the ladder, starting with practices + that are cheap but have a relatively higher chance of producing false + negatives, and proceeding to those that have higher computational + cost and lower risk of false negatives. + + +#### [6.2.1](#section-6.2.1). Simple String Comparison + + If two URIs, when considered as character strings, are identical, + then it is safe to conclude that they are equivalent. This type of + equivalence test has very low computational cost and is in wide use + in a variety of applications, particularly in the domain of parsing. + + Testing strings for equivalence requires some basic precautions. + This procedure is often referred to as "bit-for-bit" or + "byte-for-byte" comparison, which is potentially misleading. Testing + strings for equality is normally based on pair comparison of the + characters that make up the strings, starting from the first and + proceeding until both strings are exhausted and all characters are + found to be equal, until a pair of characters compares unequal, or + until one of the strings is exhausted before the other. + + This character comparison requires that each pair of characters be + put in comparable form. For example, should one URI be stored in a + byte array in EBCDIC encoding and the second in a Java String object + (UTF-16), bit-for-bit comparisons applied naively will produce + errors. It is better to speak of equality on a character-for- + character basis rather than on a byte-for-byte or bit-for-bit basis. + In practical terms, character-by-character comparisons should be done + codepoint-by-codepoint after conversion to a common character + encoding. + + False negatives are caused by the production and use of URI aliases. + Unnecessary aliases can be reduced, regardless of the comparison + method, by consistently providing URI references in an already- + normalized form (i.e., a form identical to what would be produced + after normalization is applied, as described below). + + + + +Berners-Lee, et al. Standards Track [Page 39]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + Protocols and data formats often limit some URI comparisons to simple + string comparison, based on the theory that people and + implementations will, in their own best interest, be consistent in + providing URI references, or at least consistent enough to negate any + efficiency that might be obtained from further normalization. + + +#### [6.2.2](#section-6.2.2). Syntax-Based Normalization + + Implementations may use logic based on the definitions provided by + this specification to reduce the probability of false negatives. + This processing is moderately higher in cost than character-for- + character string comparison. For example, an application using this + approach could reasonably consider the following two URIs equivalent: + + example://a/b/c/%7Bfoo%7D + eXAMPLE://a/./b/../b/%63/%7bfoo%7d + + Web user agents, such as browsers, typically apply this type of URI + normalization when determining whether a cached response is + available. Syntax-based normalization includes such techniques as + case normalization, percent-encoding normalization, and removal of + dot-segments. + + +##### [6.2.2.1](#section-6.2.2.1). Case Normalization + + For all URIs, the hexadecimal digits within a percent-encoding + triplet (e.g., "%3a" versus "%3A") are case-insensitive and therefore + should be normalized to use uppercase letters for the digits A-F. + + When a URI uses components of the generic syntax, the component + syntax equivalence rules always apply; namely, that the scheme and + host are case-insensitive and therefore should be normalized to + lowercase. For example, the URI <HTTP://www.EXAMPLE.com/> is + equivalent to <http://www.example.com/>. The other generic syntax + components are assumed to be case-sensitive unless specifically + defined otherwise by the scheme (see [Section 6.2.3](#section-6.2.3)). + + +##### [6.2.2.2](#section-6.2.2.2). Percent-Encoding Normalization + + The percent-encoding mechanism ([Section 2.1](#section-2.1)) is a frequent source of + variance among otherwise identical URIs. In addition to the case + normalization issue noted above, some URI producers percent-encode + octets that do not require percent-encoding, resulting in URIs that + are equivalent to their non-encoded counterparts. These URIs should + be normalized by decoding any percent-encoded octet that corresponds + to an unreserved character, as described in [Section 2.3](#section-2.3). + + + + + +Berners-Lee, et al. Standards Track [Page 40]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +##### [6.2.2.3](#section-6.2.2.3). Path Segment Normalization + + The complete path segments "." and ".." are intended only for use + within relative references ([Section 4.1](#section-4.1)) and are removed as part of + the reference resolution process ([Section 5.2](#section-5.2)). However, some + deployed implementations incorrectly assume that reference resolution + is not necessary when the reference is already a URI and thus fail to + remove dot-segments when they occur in non-relative paths. URI + normalizers should remove dot-segments by applying the + remove_dot_segments algorithm to the path, as described in + [Section 5.2.4](#section-5.2.4). + + +#### [6.2.3](#section-6.2.3). Scheme-Based Normalization + + The syntax and semantics of URIs vary from scheme to scheme, as + described by the defining specification for each scheme. + Implementations may use scheme-specific rules, at further processing + cost, to reduce the probability of false negatives. For example, + because the "http" scheme makes use of an authority component, has a + default port of "80", and defines an empty path to be equivalent to + "/", the following four URIs are equivalent: + + http://example.com + http://example.com/ + [http://example.com:/](http://example.com:/)[http://example.com:80/](http://example.com:80/) + + In general, a URI that uses the generic syntax for authority with an + empty path should be normalized to a path of "/". Likewise, an + explicit ":port", for which the port is empty or the default for the + scheme, is equivalent to one where the port and its ":" delimiter are + elided and thus should be removed by scheme-based normalization. For + example, the second URI above is the normal form for the "http" + scheme. + + Another case where normalization varies by scheme is in the handling + of an empty authority component or empty host subcomponent. For many + scheme specifications, an empty authority or host is considered an + error; for others, it is considered equivalent to "localhost" or the + end-user's host. When a scheme defines a default for authority and a + URI reference to that default is desired, the reference should be + normalized to an empty authority for the sake of uniformity, brevity, + and internationalization. If, however, either the userinfo or port + subcomponents are non-empty, then the host should be given explicitly + even if it matches the default. + + Normalization should not remove delimiters when their associated + component is empty unless licensed to do so by the scheme + + + +Berners-Lee, et al. Standards Track [Page 41]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + specification. For example, the URI "http://example.com/?" cannot be + assumed to be equivalent to any of the examples above. Likewise, the + presence or absence of delimiters within a userinfo subcomponent is + usually significant to its interpretation. The fragment component is + not subject to any scheme-based normalization; thus, two URIs that + differ only by the suffix "#" are considered different regardless of + the scheme. + + Some schemes define additional subcomponents that consist of case- + insensitive data, giving an implicit license to normalizers to + convert this data to a common case (e.g., all lowercase). For + example, URI schemes that define a subcomponent of path to contain an + Internet hostname, such as the "mailto" URI scheme, cause that + subcomponent to be case-insensitive and thus subject to case + normalization (e.g., "mailto:Joe@Example.COM" is equivalent to + "mailto:Joe@example.com", even though the generic syntax considers + the path component to be case-sensitive). + + Other scheme-specific normalizations are possible. + + +#### [6.2.4](#section-6.2.4). Protocol-Based Normalization + + Substantial effort to reduce the incidence of false negatives is + often cost-effective for web spiders. Therefore, they implement even + more aggressive techniques in URI comparison. For example, if they + observe that a URI such as + + http://example.com/data + + redirects to a URI differing only in the trailing slash + + http://example.com/data/ + + they will likely regard the two as equivalent in the future. This + kind of technique is only appropriate when equivalence is clearly + indicated by both the result of accessing the resources and the + common conventions of their scheme's dereference algorithm (in this + case, use of redirection by HTTP origin servers to avoid problems + with relative references). + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 42]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +## [7](#section-7). Security Considerations + + A URI does not in itself pose a security threat. However, as URIs + are often used to provide a compact set of instructions for access to + network resources, care must be taken to properly interpret the data + within a URI, to prevent that data from causing unintended access, + and to avoid including data that should not be revealed in plain + text. + + +### [7.1](#section-7.1). Reliability and Consistency + + There is no guarantee that once a URI has been used to retrieve + information, the same information will be retrievable by that URI in + the future. Nor is there any guarantee that the information + retrievable via that URI in the future will be observably similar to + that retrieved in the past. The URI syntax does not constrain how a + given scheme or authority apportions its namespace or maintains it + over time. Such guarantees can only be obtained from the person(s) + controlling that namespace and the resource in question. A specific + URI scheme may define additional semantics, such as name persistence, + if those semantics are required of all naming authorities for that + scheme. + + +### [7.2](#section-7.2). Malicious Construction + + It is sometimes possible to construct a URI so that an attempt to + perform a seemingly harmless, idempotent operation, such as the + retrieval of a representation, will in fact cause a possibly damaging + remote operation. The unsafe URI is typically constructed by + specifying a port number other than that reserved for the network + protocol in question. The client unwittingly contacts a site running + a different protocol service, and data within the URI contains + instructions that, when interpreted according to this other protocol, + cause an unexpected operation. A frequent example of such abuse has + been the use of a protocol-based scheme with a port component of + "25", thereby fooling user agent software into sending an unintended + or impersonating message via an SMTP server. + + Applications should prevent dereference of a URI that specifies a TCP + port number within the "well-known port" range (0 - 1023) unless the + protocol being used to dereference that URI is compatible with the + protocol expected on that well-known port. Although IANA maintains a + registry of well-known ports, applications should make such + restrictions user-configurable to avoid preventing the deployment of + new services. + + + + + + +Berners-Lee, et al. Standards Track [Page 43]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + When a URI contains percent-encoded octets that match the delimiters + for a given resolution or dereference protocol (for example, CR and + LF characters for the TELNET protocol), these percent-encodings must + not be decoded before transmission across that protocol. Transfer of + the percent-encoding, which might violate the protocol, is less + harmful than allowing decoded octets to be interpreted as additional + operations or parameters, perhaps triggering an unexpected and + possibly harmful remote operation. + + +### [7.3](#section-7.3). Back-End Transcoding + + When a URI is dereferenced, the data within it is often parsed by + both the user agent and one or more servers. In HTTP, for example, a + typical user agent will parse a URI into its five major components, + access the authority's server, and send it the data within the + authority, path, and query components. A typical server will take + that information, parse the path into segments and the query into + key/value pairs, and then invoke implementation-specific handlers to + respond to the request. As a result, a common security concern for + server implementations that handle a URI, either as a whole or split + into separate components, is proper interpretation of the octet data + represented by the characters and percent-encodings within that URI. + + Percent-encoded octets must be decoded at some point during the + dereference process. Applications must split the URI into its + components and subcomponents prior to decoding the octets, as + otherwise the decoded octets might be mistaken for delimiters. + Security checks of the data within a URI should be applied after + decoding the octets. Note, however, that the "%00" percent-encoding + (NUL) may require special handling and should be rejected if the + application is not expecting to receive raw data within a component. + + Special care should be taken when the URI path interpretation process + involves the use of a back-end file system or related system + functions. File systems typically assign an operational meaning to + special characters, such as the "/", "\", ":", "[", and "]" + characters, and to special device names like ".", "..", "...", "aux", + "lpt", etc. In some cases, merely testing for the existence of such + a name will cause the operating system to pause or invoke unrelated + system calls, leading to significant security concerns regarding + denial of service and unintended data transfer. It would be + impossible for this specification to list all such significant + characters and device names. Implementers should research the + reserved names and characters for the types of storage device that + may be attached to their applications and restrict the use of data + obtained from URI components accordingly. + + + + + +Berners-Lee, et al. Standards Track [Page 44]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +### [7.4](#section-7.4). Rare IP Address Formats + + Although the URI syntax for IPv4address only allows the common + dotted-decimal form of IPv4 address literal, many implementations + that process URIs make use of platform-dependent system routines, + such as gethostbyname() and inet_aton(), to translate the string + literal to an actual IP address. Unfortunately, such system routines + often allow and process a much larger set of formats than those + described in [Section 3.2.2](#section-3.2.2). + + For example, many implementations allow dotted forms of three + numbers, wherein the last part is interpreted as a 16-bit quantity + and placed in the right-most two bytes of the network address (e.g., + a Class B network). Likewise, a dotted form of two numbers means + that the last part is interpreted as a 24-bit quantity and placed in + the right-most three bytes of the network address (Class A), and a + single number (without dots) is interpreted as a 32-bit quantity and + stored directly in the network address. Adding further to the + confusion, some implementations allow each dotted part to be + interpreted as decimal, octal, or hexadecimal, as specified in the C + language (i.e., a leading 0x or 0X implies hexadecimal; a leading 0 + implies octal; otherwise, the number is interpreted as decimal). + + These additional IP address formats are not allowed in the URI syntax + due to differences between platform implementations. However, they + can become a security concern if an application attempts to filter + access to resources based on the IP address in string literal format. + If this filtering is performed, literals should be converted to + numeric form and filtered based on the numeric value, and not on a + prefix or suffix of the string form. + + +### [7.5](#section-7.5). Sensitive Information + + URI producers should not provide a URI that contains a username or + password that is intended to be secret. URIs are frequently + displayed by browsers, stored in clear text bookmarks, and logged by + user agent history and intermediary applications (proxies). A + password appearing within the userinfo component is deprecated and + should be considered an error (or simply ignored) except in those + rare cases where the 'password' parameter is intended to be public. + + +### [7.6](#section-7.6). Semantic Attacks + + Because the userinfo subcomponent is rarely used and appears before + the host in the authority component, it can be used to construct a + URI intended to mislead a human user by appearing to identify one + (trusted) naming authority while actually identifying a different + authority hidden behind the noise. For example + + + +Berners-Lee, et al. Standards Track [Page 45]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + ftp://cnn.example.com&story=breaking_news@10.0.0.1/top_story.htm + + might lead a human user to assume that the host is 'cnn.example.com', + whereas it is actually '10.0.0.1'. Note that a misleading userinfo + subcomponent could be much longer than the example above. + + A misleading URI, such as that above, is an attack on the user's + preconceived notions about the meaning of a URI rather than an attack + on the software itself. User agents may be able to reduce the impact + of such attacks by distinguishing the various components of the URI + when they are rendered, such as by using a different color or tone to + render userinfo if any is present, though there is no panacea. More + information on URI-based semantic attacks can be found in [[Siedzik](#ref-Siedzik)]. + + +## [8](#section-8). IANA Considerations + + URI scheme names, as defined by <scheme> in [Section 3.1](#section-3.1), form a + registered namespace that is managed by IANA according to the + procedures defined in [[BCP35](#ref-BCP35)]. No IANA actions are required by this + document. + + +## [9](#section-9). Acknowledgements + + This specification is derived from [RFC 2396](/doc/html/rfc2396) [[RFC2396](/doc/html/rfc2396)], [RFC 1808](/doc/html/rfc1808) + [[RFC1808](/doc/html/rfc1808)], and [RFC 1738](/doc/html/rfc1738) [[RFC1738](/doc/html/rfc1738)]; the acknowledgements in those + documents still apply. It also incorporates the update (with + corrections) for IPv6 literals in the host syntax, as defined by + Robert M. Hinden, Brian E. Carpenter, and Larry Masinter in + [[RFC2732](/doc/html/rfc2732)]. In addition, contributions by Gisle Aas, Reese Anschultz, + Daniel Barclay, Tim Bray, Mike Brown, Rob Cameron, Jeremy Carroll, + Dan Connolly, Adam M. Costello, John Cowan, Jason Diamond, Martin + Duerst, Stefan Eissing, Clive D.W. Feather, Al Gilman, Tony Hammond, + Elliotte Harold, Pat Hayes, Henry Holtzman, Ian B. Jacobs, Michael + Kay, John C. Klensin, Graham Klyne, Dan Kohn, Bruce Lilly, Andrew + Main, Dave McAlpin, Ira McDonald, Michael Mealling, Ray Merkert, + Stephen Pollei, Julian Reschke, Tomas Rokicki, Miles Sabin, Kai + Schaetzl, Mark Thomson, Ronald Tschalaer, Norm Walsh, Marc Warne, + Stuart Williams, and Henry Zongaro are gratefully acknowledged. + + +## [10](#section-10). References +### [10.1](#section-10.1). Normative References + + [[ASCII](No href)] American National Standards Institute, "Coded Character + Set -- 7-bit American Standard Code for Information + Interchange", ANSI X3.4, 1986. + + + + + +Berners-Lee, et al. Standards Track [Page 46]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + [[RFC2234](No href)] Crocker, D. and P. Overell, "Augmented BNF for Syntax + Specifications: ABNF", [RFC 2234](/doc/html/rfc2234), November 1997. + + [[STD63](No href)] Yergeau, F., "UTF-8, a transformation format of + ISO 10646", STD 63, [RFC 3629](/doc/html/rfc3629), November 2003. + + [[UCS](No href)] International Organization for Standardization, + "Information Technology - Universal Multiple-Octet Coded + Character Set (UCS)", ISO/IEC 10646:2003, December 2003. + + +### [10.2](#section-10.2). Informative References + + [[BCP19](No href)] Freed, N. and J. Postel, "IANA Charset Registration + Procedures", [BCP 19](/doc/html/bcp19), [RFC 2978](/doc/html/rfc2978), October 2000. + + [[BCP35](No href)] Petke, R. and I. King, "Registration Procedures for URL + Scheme Names", [BCP 35](/doc/html/bcp35), [RFC 2717](/doc/html/rfc2717), November 1999. + + [[RFC0952](No href)] Harrenstien, K., Stahl, M., and E. Feinler, "DoD Internet + host table specification", [RFC 952](/doc/html/rfc952), October 1985. + + [[RFC1034](No href)] Mockapetris, P., "Domain names - concepts and facilities", + STD 13, [RFC 1034](/doc/html/rfc1034), November 1987. + + [[RFC1123](No href)] Braden, R., "Requirements for Internet Hosts - Application + and Support", STD 3, [RFC 1123](/doc/html/rfc1123), October 1989. + + [[RFC1535](No href)] Gavron, E., "A Security Problem and Proposed Correction + With Widely Deployed DNS Software", [RFC 1535](/doc/html/rfc1535), + October 1993. + + [[RFC1630](No href)] Berners-Lee, T., "Universal Resource Identifiers in WWW: A + Unifying Syntax for the Expression of Names and Addresses + of Objects on the Network as used in the World-Wide Web", + [RFC 1630](/doc/html/rfc1630), June 1994. + + [[RFC1736](No href)] Kunze, J., "Functional Recommendations for Internet + Resource Locators", [RFC 1736](/doc/html/rfc1736), February 1995. + + [[RFC1737](No href)] Sollins, K. and L. Masinter, "Functional Requirements for + Uniform Resource Names", [RFC 1737](/doc/html/rfc1737), December 1994. + + [[RFC1738](No href)] Berners-Lee, T., Masinter, L., and M. McCahill, "Uniform + Resource Locators (URL)", [RFC 1738](/doc/html/rfc1738), December 1994. + + [[RFC1808](No href)] Fielding, R., "Relative Uniform Resource Locators", + [RFC 1808](/doc/html/rfc1808), June 1995. + + + + +Berners-Lee, et al. Standards Track [Page 47]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + [[RFC2046](No href)] Freed, N. and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part Two: Media Types", [RFC 2046](/doc/html/rfc2046), + November 1996. + + [[RFC2141](No href)] Moats, R., "URN Syntax", [RFC 2141](/doc/html/rfc2141), May 1997. + + [[RFC2396](No href)] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform + Resource Identifiers (URI): Generic Syntax", [RFC 2396](/doc/html/rfc2396), + August 1998. + + [[RFC2518](No href)] Goland, Y., Whitehead, E., Faizi, A., Carter, S., and D. + Jensen, "HTTP Extensions for Distributed Authoring -- + WEBDAV", [RFC 2518](/doc/html/rfc2518), February 1999. + + [[RFC2557](No href)] Palme, J., Hopmann, A., and N. Shelness, "MIME + Encapsulation of Aggregate Documents, such as HTML + (MHTML)", [RFC 2557](/doc/html/rfc2557), March 1999. + + [[RFC2718](No href)] Masinter, L., Alvestrand, H., Zigmond, D., and R. Petke, + "Guidelines for new URL Schemes", [RFC 2718](/doc/html/rfc2718), November 1999. + + [[RFC2732](No href)] Hinden, R., Carpenter, B., and L. Masinter, "Format for + Literal IPv6 Addresses in URL's", [RFC 2732](/doc/html/rfc2732), December 1999. + + [[RFC3305](No href)] Mealling, M. and R. Denenberg, "Report from the Joint + W3C/IETF URI Planning Interest Group: Uniform Resource + Identifiers (URIs), URLs, and Uniform Resource Names + (URNs): Clarifications and Recommendations", [RFC 3305](/doc/html/rfc3305), + August 2002. + + [[RFC3490](No href)] Faltstrom, P., Hoffman, P., and A. Costello, + "Internationalizing Domain Names in Applications (IDNA)", + [RFC 3490](/doc/html/rfc3490), March 2003. + + [[RFC3513](No href)] Hinden, R. and S. Deering, "Internet Protocol Version 6 + (IPv6) Addressing Architecture", [RFC 3513](/doc/html/rfc3513), April 2003. + + [[Siedzik](No href)] Siedzik, R., "Semantic Attacks: What's in a URL?", + April 2001, <[http://www.giac.org/practical/gsec/](https://www.giac.org/practical/gsec/Richard_Siedzik_GSEC.pdf)[Richard_Siedzik_GSEC.pdf](https://www.giac.org/practical/gsec/Richard_Siedzik_GSEC.pdf)>. + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 48]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +## [Appendix A](#appendix-A). Collected ABNF for URI + + URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ] + + hier-part = "//" authority path-abempty + / path-absolute + / path-rootless + / path-empty + + URI-reference = URI / relative-ref + + absolute-URI = scheme ":" hier-part [ "?" query ] + + relative-ref = relative-part [ "?" query ] [ "#" fragment ] + + relative-part = "//" authority path-abempty + / path-absolute + / path-noscheme + / path-empty + + scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) + + authority = [ userinfo "@" ] host [ ":" port ] + userinfo = *( unreserved / pct-encoded / sub-delims / ":" ) + host = IP-literal / IPv4address / reg-name + port = *DIGIT + + IP-literal = "[" ( IPv6address / IPvFuture ) "]" + + IPvFuture = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" ) + + IPv6address = 6( h16 ":" ) ls32 + / "::" 5( h16 ":" ) ls32 + / [ h16 ] "::" 4( h16 ":" ) ls32 + / [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32 + / [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32 + / [ *3( h16 ":" ) h16 ] "::" h16 ":" ls32 + / [ *4( h16 ":" ) h16 ] "::" ls32 + / [ *5( h16 ":" ) h16 ] "::" h16 + / [ *6( h16 ":" ) h16 ] "::" + + h16 = 1*4HEXDIG + ls32 = ( h16 ":" h16 ) / IPv4address + IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet + + + + + + + +Berners-Lee, et al. Standards Track [Page 49]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + dec-octet = DIGIT ; 0-9 + / %x31-39 DIGIT ; 10-99 + / "1" 2DIGIT ; 100-199 + / "2" %x30-34 DIGIT ; 200-249 + / "25" %x30-35 ; 250-255 + + reg-name = *( unreserved / pct-encoded / sub-delims ) + + path = path-abempty ; begins with "/" or is empty + / path-absolute ; begins with "/" but not "//" + / path-noscheme ; begins with a non-colon segment + / path-rootless ; begins with a segment + / path-empty ; zero characters + + path-abempty = *( "/" segment ) + path-absolute = "/" [ segment-nz *( "/" segment ) ] + path-noscheme = segment-nz-nc *( "/" segment ) + path-rootless = segment-nz *( "/" segment ) + path-empty = 0<pchar> + + segment = *pchar + segment-nz = 1*pchar + segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" ) + ; non-zero-length segment without any colon ":" + + pchar = unreserved / pct-encoded / sub-delims / ":" / "@" + + query = *( pchar / "/" / "?" ) + + fragment = *( pchar / "/" / "?" ) + + pct-encoded = "%" HEXDIG HEXDIG + + unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" + reserved = gen-delims / sub-delims + gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@" + sub-delims = "!" / "$" / "&" / "'" / "(" / ")" + / "*" / "+" / "," / ";" / "=" + + +## [Appendix B](#appendix-B). Parsing a URI Reference with a Regular Expression + + As the "first-match-wins" algorithm is identical to the "greedy" + disambiguation method used by POSIX regular expressions, it is + natural and commonplace to use a regular expression for parsing the + potential five components of a URI reference. + + The following line is the regular expression for breaking-down a + well-formed URI reference into its components. + + + +Berners-Lee, et al. Standards Track [Page 50]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? + 12 3 4 5 6 7 8 9 + + The numbers in the second line above are only to assist readability; + they indicate the reference points for each subexpression (i.e., each + paired parenthesis). We refer to the value matched for subexpression + <n> as $<n>. For example, matching the above expression to + + [http://www.ics.uci.edu/pub/ietf/uri/#Related](http://www.ics.uci.edu/pub/ietf/uri/#Related) + + results in the following subexpression matches: + + $1 = http: + $2 = http + $3 = //www.ics.uci.edu + $4 = www.ics.uci.edu + $5 = /pub/ietf/uri/ + $6 = <undefined> + $7 = <undefined> + $8 = #Related + $9 = Related + + where <undefined> indicates that the component is not present, as is + the case for the query component in the above example. Therefore, we + can determine the value of the five components as + + scheme = $2 + authority = $4 + path = $5 + query = $7 + fragment = $9 + + Going in the opposite direction, we can recreate a URI reference from + its components by using the algorithm of [Section 5.3](#section-5.3). + + +## [Appendix C](#appendix-C). Delimiting a URI in Context + + URIs are often transmitted through formats that do not provide a + clear context for their interpretation. For example, there are many + occasions when a URI is included in plain text; examples include text + sent in email, USENET news, and on printed paper. In such cases, it + is important to be able to delimit the URI from the rest of the text, + and in particular from punctuation marks that might be mistaken for + part of the URI. + + In practice, URIs are delimited in a variety of ways, but usually + within double-quotes "http://example.com/", angle brackets + <http://example.com/>, or just by using whitespace: + + + +Berners-Lee, et al. Standards Track [Page 51]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + http://example.com/ + + These wrappers do not form part of the URI. + + In some cases, extra whitespace (spaces, line-breaks, tabs, etc.) may + have to be added to break a long URI across lines. The whitespace + should be ignored when the URI is extracted. + + No whitespace should be introduced after a hyphen ("-") character. + Because some typesetters and printers may (erroneously) introduce a + hyphen at the end of line when breaking it, the interpreter of a URI + containing a line break immediately after a hyphen should ignore all + whitespace around the line break and should be aware that the hyphen + may or may not actually be part of the URI. + + Using <> angle brackets around each URI is especially recommended as + a delimiting style for a reference that contains embedded whitespace. + + The prefix "URL:" (with or without a trailing space) was formerly + recommended as a way to help distinguish a URI from other bracketed + designators, though it is not commonly used in practice and is no + longer recommended. + + For robustness, software that accepts user-typed URI should attempt + to recognize and strip both delimiters and embedded whitespace. + + For example, the text + + Yes, Jim, I found it under "http://www.w3.org/Addressing/", + but you can probably pick it up from <ftp://foo.example. + com/rfc/>. Note the warning in <[http://www.ics.uci.edu/pub/](http://www.ics.uci.edu/pub/ietf/uri/historical.html#WARNING)[ietf/uri/historical.html#WARNING](http://www.ics.uci.edu/pub/ietf/uri/historical.html#WARNING)>. + + contains the URI references + + [http://www.w3.org/Addressing/](http://www.w3.org/Addressing/) + ftp://foo.example.com/rfc/ + [http://www.ics.uci.edu/pub/ietf/uri/historical.html#WARNING](http://www.ics.uci.edu/pub/ietf/uri/historical.html#WARNING)Berners-Lee, et al. Standards Track [Page 52]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 +## [Appendix D](#appendix-D). Changes from [RFC 2396](/doc/html/rfc2396) +### [D.1](#appendix-D.1). Additions + + An ABNF rule for URI has been introduced to correspond to one common + usage of the term: an absolute URI with optional fragment. + + IPv6 (and later) literals have been added to the list of possible + identifiers for the host portion of an authority component, as + described by [[RFC2732](/doc/html/rfc2732)], with the addition of "[" and "]" to the + reserved set and a version flag to anticipate future versions of IP + literals. Square brackets are now specified as reserved within the + authority component and are not allowed outside their use as + delimiters for an IP literal within host. In order to make this + change without changing the technical definition of the path, query, + and fragment components, those rules were redefined to directly + specify the characters allowed. + + As [[RFC2732](/doc/html/rfc2732)] defers to [[RFC3513](/doc/html/rfc3513)] for definition of an IPv6 literal + address, which, unfortunately, lacks an ABNF description of + IPv6address, we created a new ABNF rule for IPv6address that matches + the text representations defined by [Section 2.2 of [RFC3513]](/doc/html/rfc3513#section-2.2). + Likewise, the definition of IPv4address has been improved in order to + limit each decimal octet to the range 0-255. + + [Section 6](#section-6), on URI normalization and comparison, has been completely + rewritten and extended by using input from Tim Bray and discussion + within the W3C Technical Architecture Group. + + +### [D.2](#appendix-D.2). Modifications + + The ad-hoc BNF syntax of [RFC 2396](/doc/html/rfc2396) has been replaced with the ABNF of + [[RFC2234](/doc/html/rfc2234)]. This change required all rule names that formerly + included underscore characters to be renamed with a dash instead. In + addition, a number of syntax rules have been eliminated or simplified + to make the overall grammar more comprehensible. Specifications that + refer to the obsolete grammar rules may be understood by replacing + those rules according to the following table: + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 53]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + +----------------+--------------------------------------------------+ + | obsolete rule | translation | + +----------------+--------------------------------------------------+ + | absoluteURI | absolute-URI | + | relativeURI | relative-part [ "?" query ] | + | hier_part | ( "//" authority path-abempty / | + | | path-absolute ) [ "?" query ] | + | | | + | opaque_part | path-rootless [ "?" query ] | + | net_path | "//" authority path-abempty | + | abs_path | path-absolute | + | rel_path | path-rootless | + | rel_segment | segment-nz-nc | + | reg_name | reg-name | + | server | authority | + | hostport | host [ ":" port ] | + | hostname | reg-name | + | path_segments | path-abempty | + | param | *<pchar excluding ";"> | + | | | + | uric | unreserved / pct-encoded / ";" / "?" / ":" | + | | / "@" / "&" / "=" / "+" / "$" / "," / "/" | + | | | + | uric_no_slash | unreserved / pct-encoded / ";" / "?" / ":" | + | | / "@" / "&" / "=" / "+" / "$" / "," | + | | | + | mark | "-" / "_" / "." / "!" / "~" / "*" / "'" | + | | / "(" / ")" | + | | | + | escaped | pct-encoded | + | hex | HEXDIG | + | alphanum | ALPHA / DIGIT | + +----------------+--------------------------------------------------+ + + Use of the above obsolete rules for the definition of scheme-specific + syntax is deprecated. + + [Section 2](#section-2), on characters, has been rewritten to explain what + characters are reserved, when they are reserved, and why they are + reserved, even when they are not used as delimiters by the generic + syntax. The mark characters that are typically unsafe to decode, + including the exclamation mark ("!"), asterisk ("*"), single-quote + ("'"), and open and close parentheses ("(" and ")"), have been moved + to the reserved set in order to clarify the distinction between + reserved and unreserved and, hopefully, to answer the most common + question of scheme designers. Likewise, the section on + percent-encoded characters has been rewritten, and URI normalizers + are now given license to decode any percent-encoded octets + + + +Berners-Lee, et al. Standards Track [Page 54]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + corresponding to unreserved characters. In general, the terms + "escaped" and "unescaped" have been replaced with "percent-encoded" + and "decoded", respectively, to reduce confusion with other forms of + escape mechanisms. + + The ABNF for URI and URI-reference has been redesigned to make them + more friendly to LALR parsers and to reduce complexity. As a result, + the layout form of syntax description has been removed, along with + the uric, uric_no_slash, opaque_part, net_path, abs_path, rel_path, + path_segments, rel_segment, and mark rules. All references to + "opaque" URIs have been replaced with a better description of how the + path component may be opaque to hierarchy. The relativeURI rule has + been replaced with relative-ref to avoid unnecessary confusion over + whether they are a subset of URI. The ambiguity regarding the + parsing of URI-reference as a URI or a relative-ref with a colon in + the first segment has been eliminated through the use of five + separate path matching rules. + + The fragment identifier has been moved back into the section on + generic syntax components and within the URI and relative-ref rules, + though it remains excluded from absolute-URI. The number sign ("#") + character has been moved back to the reserved set as a result of + reintegrating the fragment syntax. + + The ABNF has been corrected to allow the path component to be empty. + This also allows an absolute-URI to consist of nothing after the + "scheme:", as is present in practice with the "dav:" namespace + [[RFC2518](/doc/html/rfc2518)] and with the "about:" scheme used internally by many WWW + browser implementations. The ambiguity regarding the boundary + between authority and path has been eliminated through the use of + five separate path matching rules. + + Registry-based naming authorities that use the generic syntax are now + defined within the host rule. This change allows current + implementations, where whatever name provided is simply fed to the + local name resolution mechanism, to be consistent with the + specification. It also removes the need to re-specify DNS name + formats here. Furthermore, it allows the host component to contain + percent-encoded octets, which is necessary to enable + internationalized domain names to be provided in URIs, processed in + their native character encodings at the application layers above URI + processing, and passed to an IDNA library as a registered name in the + UTF-8 character encoding. The server, hostport, hostname, + domainlabel, toplabel, and alphanum rules have been removed. + + The resolving relative references algorithm of [[RFC2396](/doc/html/rfc2396)] has been + rewritten with pseudocode for this revision to improve clarity and + fix the following issues: + + + +Berners-Lee, et al. Standards Track [Page 55]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + o [[RFC2396] section 5.2](/doc/html/rfc2396#section-5.2), step 6a, failed to account for a base URI + with no path. + + o Restored the behavior of [[RFC1808](/doc/html/rfc1808)] where, if the reference + contains an empty path and a defined query component, the target + URI inherits the base URI's path component. + + o The determination of whether a URI reference is a same-document + reference has been decoupled from the URI parser, simplifying the + URI processing interface within applications in a way consistent + with the internal architecture of deployed URI processing + implementations. The determination is now based on comparison to + the base URI after transforming a reference to absolute form, + rather than on the format of the reference itself. This change + may result in more references being considered "same-document" + under this specification than there would be under the rules given + in [RFC 2396](/doc/html/rfc2396), especially when normalization is used to reduce + aliases. However, it does not change the status of existing + same-document references. + + o Separated the path merge routine into two routines: merge, for + describing combination of the base URI path with a relative-path + reference, and remove_dot_segments, for describing how to remove + the special "." and ".." segments from a composed path. The + remove_dot_segments algorithm is now applied to all URI reference + paths in order to match common implementations and to improve the + normalization of URIs in practice. This change only impacts the + parsing of abnormal references and same-scheme references wherein + the base URI has a non-hierarchical path. + +Index + + A + ABNF 11 + absolute 27 + absolute-path 26 + absolute-URI 27 + access 9 + authority 17, 18 + + B + base URI 28 + + C + character encoding 4 + character 4 + characters 8, 11 + coded character set 4 + + + +Berners-Lee, et al. Standards Track [Page 56]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + D + dec-octet 20 + dereference 9 + dot-segments 23 + + F + fragment 16, 24 + + G + gen-delims 13 + generic syntax 6 + + H + h16 20 + hier-part 16 + hierarchical 10 + host 18 + + I + identifier 5 + IP-literal 19 + IPv4 20 + IPv4address 19, 20 + IPv6 19 + IPv6address 19, 20 + IPvFuture 19 + + L + locator 7 + ls32 20 + + M + merge 32 + + N + name 7 + network-path 26 + + P + path 16, 22, 26 + path-abempty 22 + path-absolute 22 + path-empty 22 + path-noscheme 22 + path-rootless 22 + path-abempty 16, 22, 26 + path-absolute 16, 22, 26 + path-empty 16, 22, 26 + + + +Berners-Lee, et al. Standards Track [Page 57]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + path-rootless 16, 22 + pchar 23 + pct-encoded 12 + percent-encoding 12 + port 22 + + Q + query 16, 23 + + R + reg-name 21 + registered name 20 + relative 10, 28 + relative-path 26 + relative-ref 26 + remove_dot_segments 33 + representation 9 + reserved 12 + resolution 9, 28 + resource 5 + retrieval 9 + + S + same-document 27 + sameness 9 + scheme 16, 17 + segment 22, 23 + segment-nz 23 + segment-nz-nc 23 + sub-delims 13 + suffix 27 + + T + transcription 8 + + U + uniform 4 + unreserved 13 + URI grammar + absolute-URI 27 + ALPHA 11 + authority 18 + CR 11 + dec-octet 20 + DIGIT 11 + DQUOTE 11 + fragment 24 + gen-delims 13 + + + +Berners-Lee, et al. Standards Track [Page 58]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + + h16 20 + HEXDIG 11 + hier-part 16 + host 19 + IP-literal 19 + IPv4address 20 + IPv6address 20 + IPvFuture 19 + LF 11 + ls32 20 + OCTET 11 + path 22 + path-abempty 22 + path-absolute 22 + path-empty 22 + path-noscheme 22 + path-rootless 22 + pchar 23 + pct-encoded 12 + port 22 + query 24 + reg-name 21 + relative-ref 26 + reserved 13 + scheme 17 + segment 23 + segment-nz 23 + segment-nz-nc 23 + SP 11 + sub-delims 13 + unreserved 13 + URI 16 + URI-reference 25 + userinfo 18 + URI 16 + URI-reference 25 + URL 7 + URN 7 + userinfo 18 + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 59]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + +Authors' Addresses + + Tim Berners-Lee + World Wide Web Consortium + Massachusetts Institute of Technology + 77 Massachusetts Avenue + Cambridge, MA 02139 + USA + + Phone: +1-617-253-5702 + Fax: +1-617-258-5999 + EMail: timbl@w3.org + URI: [http://www.w3.org/People/Berners-Lee/](http://www.w3.org/People/Berners-Lee/) + + + Roy T. Fielding + Day Software + 5251 California Ave., Suite 110 + Irvine, CA 92617 + USA + + Phone: +1-949-679-2960 + Fax: +1-949-679-2972 + EMail: fielding@gbiv.com + URI: [http://roy.gbiv.com/](http://roy.gbiv.com/) + + + Larry Masinter + Adobe Systems Incorporated + 345 Park Ave + San Jose, CA 95110 + USA + + Phone: +1-408-536-3024 + EMail: LMM@acm.org + URI: [http://larry.masinter.net/](http://larry.masinter.net/)Berners-Lee, et al. Standards Track [Page 60]`` +__________________________________ +[RFC 3986](/doc/html/rfc3986) URI Generic Syntax January 2005 + + +Full Copyright Statement + + Copyright (C) The Internet Society (2005). + + This document is subject to the rights, licenses and restrictions + contained in [BCP 78](/doc/html/bcp78), and except as set forth therein, the authors + retain all their rights. + + This document and the information contained herein are provided on an + "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS + OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET + ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, + INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE + INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED + WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Intellectual Property + + The IETF takes no position regarding the validity or scope of any + Intellectual Property Rights or other rights that might be claimed to + pertain to the implementation or use of the technology described in + this document or the extent to which any license under such rights + might or might not be available; nor does it represent that it has + made any independent effort to identify any such rights. Information + on the IETF's procedures with respect to rights in IETF Documents can + be found in [BCP 78](/doc/html/bcp78) and [BCP 79](/doc/html/bcp79). + + Copies of IPR disclosures made to the IETF Secretariat and any + assurances of licenses to be made available, or the result of an + attempt made to obtain a general license or permission for the use of + such proprietary rights by implementers or users of this + specification can be obtained from the IETF on-line IPR repository at + [http://www.ietf.org/ipr](http://www.ietf.org/ipr). + + The IETF invites any interested party to bring to its attention any + copyrights, patents or patent applications, or other proprietary + rights that may cover technology that may be required to implement + this standard. Please address the information to the IETF at ietf- + ipr@ietf.org. + + +Acknowledgement + + Funding for the RFC Editor function is currently provided by the + Internet Society. + + + + + + +Berners-Lee, et al. Standards Track [Page 61] +`[Datatracker](/doc/rfc3986/) + + RFC 3986 +RFC - Internet Standard + + -//Info + -//Contents + -//Prefs + + + Document Document type | | RFC - Internet Standard +January 2005 +[ View errata ](https://www.rfc-editor.org/errata_search.php?rfc=3986)[ Report errata ](https://www.rfc-editor.org/errata.php#reportnew)Updated by [RFC 8820](/doc/html/rfc8820), [RFC 7320](/doc/html/rfc7320)Obsoletes [RFC 1808](/doc/html/rfc1808), [RFC 2732](/doc/html/rfc2732), [RFC 2396](/doc/html/rfc2396)Updates [RFC 1738](/doc/html/rfc1738) Was [draft-fielding-uri-rfc2396bis](/doc/draft-fielding-uri-rfc2396bis/07/) (individual in gen area) +| Select version | | + -[ 07 ](/doc/html/draft-fielding-uri-rfc2396bis-07) + -[ RFC 3986 ](/doc/html/rfc3986) + + + +| Compare versions | | +[FORM(No id) Side-by-side Inline ](get https://author-tools.ietf.org/iddiff) +| Authors | | [T. Berners-Lee](/person/timbl@w3.org)[//](mailto:timbl%40w3.org), [R. Fielding](/person/fielding@gbiv.com)[//](mailto:fielding%40gbiv.com), [L. Masinter](/person/lmm@acm.org)[//](mailto:lmm%40acm.org) +[Email authors](mailto:rfc3986@ietf.org?subject=rfc3986) +| RFC stream | | [img:IETF Logo](https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor-white.svg)[img:IETF Logo](https://static.ietf.org/dt/12.68.0/ietf/images/ietf-logo-nor.svg) +| Other formats | | [// txt ](https://www.rfc-editor.org/rfc/rfc3986.txt)[// html ](https://www.rfc-editor.org/rfc/rfc3986.html)[// w/errata ](https://www.rfc-editor.org/rfc/inline-errata/rfc3986.html)[// bibtex ](/doc/rfc3986/bibtex/) +[ Report a datatracker bug //](https://github.com/ietf-tools/datatracker/issues/new/choose)Show sidebar by defaultin:sidebar Yesin:sidebar NoTab to show by defaultin:deftab //Info in:deftab //Contents HTMLization configurationin:htmlconf //HTMLize the plaintext in:htmlconf //Plaintextify the HTML Maximum font sizein:No name Page dependenciesin:pagedeps //Inline in:pagedeps //Reference Citation linksin:reflinks // Go to reference section in:reflinks //Go to linked document + + [img:](//analytics.ietf.org/matomo.php?idsite=7) diff --git a/test/urls b/test/urls @@ -0,0 +1,19 @@ + +Cool stuff I wanna look at later: +[gb dev pandoc](https://gbdev.io/pandocs) +[gbdk](https://gbdk.org/docs/api/index.html) +[timus](acm.timus.ru) +[rfc about parsing URIs](https://datatracker.ietf.org/doc/html/rfc3986) + +Random test pages: +[duck](duckduckgo.com) +[google](google.com) +[google gumbo github](https://github.com/google/gumbo-parser) +[halimede](https://halimedemf.substack.com/p/doubting-thomasine) +[hackernews](news.ycombinator.com) +[jordan reger devblog](jordanreger.com) +[markdown color](https://markdownformatting.com/color) +[rgbds gbdev docs](https://rgbds.gbdev.io/docs/v1.0.0) +[html forms w3schools](https://www.w3schools.com/html/html_forms.asp) + +my pages: [y1nz git](https://git.y1.nz), [me! (old)](waeaves.com), [me! (crunchy/new)](https://y1.nz) diff --git a/text.c b/text.c @@ -1,5 +1,5 @@ #include <ctype.h> /* isprint */ -#include <stdlib.h> /* malloc/free, rand */ +#include <stdlib.h> /* malloc/free */ #include "config.h" #include "text.h" @@ -60,7 +60,7 @@ int create_link(FileContents *contents, Link *l) { int start = l->open_square; int width = l->close_square - l->open_square + 1; - Markup *link = create_markup(start, width, COLOR_CYAN); + Markup *link = create_markup(start, width, LINK_COLOR); /* copy the link destination into link->data */ // TODO only malloc as much as necessary @@ -83,20 +83,28 @@ FileContents * read_contents(FILE *fin) { FileContents *ret = init_contents(); - char ch; + char ch, previous_ch; int i = 0, i0; Link **links = malloc(1000 * sizeof(Link *)); int link_count = 0; for (;;) { + previous_ch = ch; ch = fgetc(fin); if (ch == EOF) break; + if ( + previous_ch == '\\' && + (ch == '[' || ch == ']' || ch == '(' || ch == ')') + ) { + ret->buffer[i-1] = ch; + continue; + } + /* link parsing. */ - // TODO extract this so it's modular. Lots of room for customization here - // TODO add escape char (skip this after a \) - if (ch == '[') { + // TODO extract this so it's more modular. + if (ch == '[' && previous_ch != '\\') { link_count++; CURRENT = malloc(sizeof(Link)); CURRENT->close_square = @@ -131,7 +139,7 @@ read_contents(FILE *fin) /* for weird characters, just display them in hex + highlight. */ /* this way they're unambiguous, searchable, & comprehensible */ - Markup *highlight = create_markup(i, 3, COLOR_BLACK); + Markup *highlight = create_markup(i, 3, COLOR_RED); highlight->active = 1; /* show it in inverse */ ret->highlights[ret->highlight_count++] = highlight;
This webpage is intended to be an accessible preview of this repository. To get a fuller picture, clone it and use the git CLI.