Is an API Always Necessary for Server-Client Communication?

[fog37]

TL;DR Summary

API and sever/client communication

It is clear that a server and a client are programs communicative iwth each other using one or more protocols (HTTP, TCP, etc.)
I conceptually understand what an API is: it is like an intermediary between two programs that makes integration easy. For example, we build app A and want to connect to app B. The connection may not be easy unless app B offers an API to which to connect...

That said, does it means that in every client/software communication there is an API, another program, seating between the client and server? Or is an API just another higher level protocol? I am confused about the relation between server, client, API...

For example, could we have a server and a client communicating with each other without an intermediary API?

In a different post, I talked about writing simple client.py and server.py scripts in Python that represent a server and a client. There was no API discussion in that context....So API don't always need to be part of the server/client exchange?

Thanks!

 

[PeterDonis]

I conceptually understand what an API is: it is like an intermediary between two programs that makes integration easy.

Usually the term "API" is more specific: it means an explicitly specified interface that one program (or more commonly a library) provides so that other programs can use it. For example, operating system kernels have APIs that enable user programs to use their services.

For example, we build app A and want to connect to app B. The connection may not be easy unless app B offers an API to which to connect...

This is not the usual usage of the term "API". I would say the reason for that is that app B is probably not the one that specifies the connection protocol; a third party standard is. For example, if app B is listening for TCP connections on a certain IP address and port, app B is not defining what IP addresses and ports are or what the TCP protocol is. It is just doing the same thing every other TCP server program does.

An API for app B, if it were a web application listening on a specific IP address and port, would be more like a specification of what URLs were available for what types of requests, and what the expected responses would be.

 

[PeterDonis]

does it means that in every client/software communication there is an API, another program, seating between the client and server?

Obviously not. In your example, there is just app A and app B. There is no program C in between them.

is an API just another higher level protocol?

Not really, because an API doesn't specify how app A connects to app B; the TCP protocol and the system of IP addresses and ports does that. An API specifies what kinds of content can be exchanged once the connection is made.

 

[FactChecker]

TL;DR Summary: API and sever/client communication

It is clear that a server and a client are programs communicative iwth each other using one or more protocols (HTTP, TCP, etc.)
I conceptually understand what an API is: it is like an intermediary between two programs that makes integration easy.

It also means that there is no need to go further into the details of the programs/libraries that are being communicated with. Therefore, it isolates one program from the internal details of another.

For example, we build app A and want to connect to app B. The connection may not be easy unless app B offers an API to which to connect...

That said, does it means that in every client/software communication there is an API, another program, seating between the client and server?

Not usually. The API just defines how to communicate or use other software. It is not usually a separate program in itself. It can define the interface part of another program or library.

Or is an API just another higher level protocol?

Yes.

I am confused about the relation between server, client, API...

For example, could we have a server and a client communicating with each other without an intermediary API?

I would say that an API exists, even if it is messy and poorly defined.

In a different post, I talked about writing simple client.py and server.py scripts in Python that represent a server and a client. There was no API discussion in that context....So API don't always need to be part of the server/client exchange?

Surely, they communicate in some defined way. For better or worse, that is an API

 

[DaveC426913]

You might think of an API like an Inbox and an Outbox on the closed door of an office worker in another department.

While your more collaborative worker sitting next to you might consult with you directly, talking to you and sharing data back and forth constantly and flexibly, this worker needs his privacy, after all he is very busy and you are not his only client, and his other clients are none of your business. Nor are the machinations of his department. Heck, they might use voodoo magic to get their results, as long as it works for them, and you have access to the In/Outboxes.

He has given you a specific way that he wants the data given to him, and a specific way he will give you data when you request it or when he's done with it. This avoids him getting your crumbs and coffee spills on his desk, (which would upset all his other clients) and it ensures you get exactly what you need and only what you need from him.

What he actually does in his office in his department (with our without your data) is something you neither know nor care about, as long as its returned clean, correct and punctually.

The worker's door itself - with its Inbox and Outbox - is the interface, provided for you to interact with him.

 

[Rive]

In a different post, I talked about writing simple client.py and server.py scripts in Python that represent a server and a client. There was no API discussion in that context....

It does not worth to mystify it.
Since they do communicate (some way) there is an (unspecified) interface between them anyway.
You may invest the effort to properly specify it for later or general use => official, specified API.
It's about the way you look at/engineer the software.

 

[fog37]

Obviously not. In your example, there is just app A and app B. There is no program C in between them.


Not really, because an API doesn't specify how app A connects to app B; the TCP protocol and the system of IP addresses and ports does that. An API specifies what kinds of content can be exchanged once the connection is made.

My confusion stems from hearing people saying that they use this or that API, for example the API specific to a website server (ex: Youtube API, Twitter API)...Some people talk about building APIs which made me think that API are actual programs/code that developers write...

 

[DaveC426913]

Some people talk about building APIs which made me think that API are actual programs/code that developers write...

It's still programming and coding being written, it's just that an API's specific purpose is as an interface to some black box. You only want to expose the functionality other developers need to access your resources safely.

 

[PeterDonis]

My confusion stems from hearing people saying that they use this or that API, for example the API specific to a website server (ex: Youtube API, Twitter API)

Yes, but you don't use those APIs just to look at videos on YouTube or read posts on Twitter. (There might be Javascript code inside those websites that uses the APIs for those sites, but you aren't writing that code, you're just using the website.) You use those APIs to do things like write third-party client apps that do things like upload your own videos to YouTube or upload posts to Twitter.

Some people talk about building APIs which made me think that API are actual programs/code that developers write...

I would say that the APIs themselves are the specifications that client programs use, and the servers that those clients talk to implement the APIs.

That said, I wouldn't get too hung up on the fine details of usages of words here; terms like "API" don't have precise definitions or usages. What actually matters is what you have to do to accomplish a particular purpose. If you just want to surf the YouTube site, for example, you don't have to know or care about what your browser or the YouTube servers are doing behind the scenes. If you want to write your own app to do something like post videos to YouTube, then you have to know more of those details. The details might be called an "API" or they might be called something else, but either way it's the content of the details that matters.

 

[fog37]

In essence, it seems that an API is really a protocol, like HTTP and TCP. But the API as a protocol seems to run above the HTTP layer of communication making functions available to the code to get content from the server in a specific way, as a json file...

 

[PeterDonis]

In essence, it seems that an API is really a protocol, like HTTP and TCP.

Sort of. A protocol is much more general. APIs are specific to particular applications. Protocols are used by basically everybody.

the API as a protocol seems to run above the HTTP layer of communication

Which in turns runs above the TCP protocol layer. And those are by no means the only layers in the network stack.

making functions available to the code to get content from the server in a specific way, as a json file...

An API doesn't have to return JSON data--and even if it does, the JSON data is most likely not just served from a file, it is computed dynamically by the server. (In some cases it might be retrieved from a database.)

 

[jack action]

The term Application Programming Interface (API) is usually used in opposition to User Interface (UI) which is the space where interactions between humans and machines occur. The output of UI is formatted with graphics, sounds, and interactive components like forms, for example, the HTML format.

API is meant to serve the essential pieces of information to another program in a simplified format, for example, JSON, XML, etc.

So a server could respond with a highly formatted, human-readable, answer to an HTTP request and a simple JSON file to the same request but sent to its API instead.

 

[fog37]

The term Application Programming Interface (API) is usually used in opposition to User Interface (UI) which is the space where interactions between humans and machines occur. The output of UI is formatted with graphics, sounds, and interactive components like forms, for example, the HTML format.

API is meant to serve the essential pieces of information to another program in a simplified format, for example, JSON, XML, etc.

So a server could respond with a highly formatted, human-readable, answer to an HTTP request and a simple JSON file to the same request but sent to its API instead.

I see I see...Thank you!

Just to make all this discussing even more real, let's say I write a python script that performs some useful task....where would the API piece fit in? I guess I would use inside the script some specific functions (every programming language has different functions/procedures to interact with the same API) to request some type of information from, for example Twitter.... here we are using an existing API...

When people actually build APIs (instead of just using them), does it mean there is some website that does not have an API and we want to create one for it so we can request information from it via a program we write?

 

[PeterDonis]

let's say I write a python script that performs some useful task....where would the API piece fit in?

It depends on what your program is doing, who its intended users are, and what it needs to perform its task.

When people actually build APIs (instead of just using them), does it mean there is some website that does not have an API and we want to create one for it so we can request information from it via a program we write?

Sometimes. And sometimes not.

 

[Bosko]

In essence, it seems that an API is really a protocol,

No. API is an interface.
Check how the OSI model works. e.g. https://www.imperva.com/learn/application-security/osi-model/

like HTTP and TCP.

HTTP and TCP are not at the same level of OSI
HTTP have commands and messages for communication with the HTTP (web) server
On API level you have a function like "give me" https://www.physicsforums.com
API translate this function on a series of lower level HTTP commands:
1. Asks DNS system for IP address of www.physicsforums.com
2. Connects to an IP address using TCP ( or UDP ) connection and port 443
3. Sends GET command from the secure HTTP(S) protocol and receives HTML file
4. ... and so on

But the API as a protocol seems to run above the HTTP layer of communication making functions available to the code to get content from the server in a specific way, as a json file...

Yes. Sort of, like that.

 

[PeterDonis]

HTTP and TCP are not at the same level of OSI

HTTP itself isn't even confined to a single level of OSI; it contains elements of levels 5, 6, and 7.

 

[Bosko]

HTTP itself isn't even confined to a single level of OSI; it contains elements of levels 5, 6, and 7.

Yes, and for me, the difference between OSI levels is not so clear.
I consider as level 5 some simple protocols like Telnet, SSH, TFTP, maybe even FTP, SAMBA(SMB-windows file transfer) ...
and mostly as level 6 some more complex like the HTTP/S because it needs, for example, to get index.php then an application (level 7) have to parse that file into a HTML tree, extract links to other resources (images, css, ...) and call HTTP GET again ...
Also the group of IEEE protocols e.g. for email system SMTP, POP3 and IMAP are mostly level 6

I am wandering , if I am using a Telnet (SSH) to connect to a server or a network switch, is that should be considered as an action on level 5 , 6 or 7 of the OSI model. I am not sure. Probably all of them.

 

[PeterDonis]

the difference between OSI levels is not so clear

In levels 5, 6, and 7, yes, I think there is a lot of overlap/combination of levels in one program or protocol, with HTTP being an example, as I mentioned. The other protocols you mention would also be examples.

In the lower levels I think the separation between layers is clearer. But of course those are the levels that most people, including most programmers, never have to see or work with; they basically Just Work almost all the time.

 

[jack action]

let's say I write a python script that performs some useful task....where would the API piece fit in?

def getJSONData():
    JSONstring = ""
    """
    fetch info from database and put it in the JSON formatted string JSONstring
    """
    return JSONstring
    

def getHTMLwebPage():
    JSONdata = getJSONdata()
    HTMLstring = ""
    """
    Insert the JSONdata into an HTML template then save in HTMLstring
    """
    return HTMLstring


def getData(url):
    HTMurl = "https://example.com/get-info"
    APIurl = "https://example.com/api/get-info"
    
    if url == HTMLurl:
        return getHTMLwebPage()
    elif url == APIurl:
        return getJSONdata()

 

[Tom.G]

Hmmm... Let's try going back to the basics at a real simple level.

Say you want to write a program, subroutine, package, etc that will return the area of different shapes. The shapes being: Square, Triangle, Circle, Regular Polyhedron.
The package of course will need to know: shape, number of sides, and at least one dimension. It will then calculate the result and present the answer.

The needed information can be passed to the package in various ways depending on the language you are using.

Assembly: in Registers, on the Stack, in specific Memory addresses
BASIC: in Named Variables
Several High Level Languages: on the Stack, on a Disc in a specific file and order

The API is the document that states where the package expects the input information, in what format (integers, floats, inches, degrees, etc.), and where it will present the result.

So an API is just a description of where a package expects information, in what format, and where it returns any results in what format.

Consider an API as a dictionary definition of what a package needs where, and where the result is.

Hope this helps!

Cheers,
Tom

 

[fog37]

def getJSONData():
    JSONstring = ""
    """
    fetch info from database and put it in the JSON formatted string JSONstring
    """
    return JSONstring
   

def getHTMLwebPage():
    JSONdata = getJSONdata()
    HTMLstring = ""
    """
    Insert the JSONdata into an HTML template then save in HTMLstring
    """
    return HTMLstring


def getData(url):
    HTMurl = "https://example.com/get-info"
    APIurl = "https://example.com/api/get-info"
   
    if url == HTMLurl:
        return getHTMLwebPage()
    elif url == APIurl:
        return getJSONdata()

Thank you for the example. I am making progress here :)

In regards to the API on a web server...I have heard the term "API server"...I guess it is simply a regular web server that exposes an API.

Can a single web server have multiple APIs at the same time? I know a web server can expose multiple "endpoints" of the same API, correct? For what I understand, the endpoints are points to which we (the program we wrote) can connect to...Are endpoints a component of the same API or are are endpoints really different APIs altogether?

Exciting topic. Thank you!

 

[jack action]

Can a single web server have multiple APIs at the same time?

You can find a list of Google's API here.

For example, from the previous list, you can find the details about the Gmail API here. The Gmail API is accessible with the URL https://gmail.googleapis.com. If you want to list the messages from a user, you use the URL https://gmail.googleapis.com/gmail/v1/users/{userId}/messages. If successful, you will get the info with the following JSON representation:

{
  "messages": [
    {
      "id": string,
      "threadId": string
    },
    ...
  ],
  "nextPageToken": string,
  "resultSizeEstimate": integer
}

Then you can get more details about each message with the messages.get service found at the URL https://gmail.googleapis.com/gmail/v1/users/{userId}/messages/{id}.

This is the same information you would get if you accessed the user account through https://mail.google.com/ but without all the extra HTML formatting meant for a browser, i.e. the User Interface (UI). The output of the Gmail API is meant to be read by some application program - as opposed to a human user - hence the name Application Programming Interface.

 

[jack action]

 

[fog37]

View attachment 340779​

Thank you @jack action...

I am appreciating more and more what API do and how ubiquitous they are...There are physical APIs and software APIs (which we are discussing). Anything that abstracts aways something else and let's us use it without worrying about the underlying functionalities and details represents an API.

• For example, the knobs on a radio are APIs....The power outlet is example of a physical outlet (we get electricity from it and don't care about all the details about the power plant and transmission lines). Inside a computer, I am sure some there are many software APIs that let components communicate with lower-level hardware. In programming, a library is sometimes called an API (ex: Keras is an API for Tensorflow, Seaborn is an API for matplotlib, etc. since they allow us to use those libraries/frameworks in a more convenient way...). Even a simple python function that converts a string from lowercase to uppercase is an API in a sense since we simply call the function $upper()$ and it does its job but we have no idea of how happens under the hood.

Back to web APIs, let's review the big picture: we have remote server and a client. The client makes a request using HTTP (a protocol that seats on top of TCP). The server responds providing the "resource". The resource can mean a lot of things. It can be a webpage, some data, or even a service (ex: we send data to be processed on the server and sent back to us). However, I realize that not all client-server communications involve an API...For example, the typical request of a webpage does not involve the browser (client) to interact with an API. I think the browser interacts directly with the server...

We have said that an API is really a set of rule, functions (sort of a protocol) for two programs to communicate and exchange information/resources. The final recipient of the requested information may be a computer program but also a human user (even if an API is not a user interface UI). Thinking of a hypothetical app on our phone, the app probably uses an API to get music files stored on a remote web server and then let's the human user listen to the music (Shazam probably works like that) so the end beneficiary in this case is the human...

• However, as ubiquitous APIs are, I read that there are many scenarios in which they are not used (see this interesting quora reply: https://www.quora.com/Do-all-websites-use-an-API-I-believe-they-would-as-a-client-needs-to-communicate-data-and-requests-to-the-server-but-is-there-an-alternative-method#:~:text=Not%20all%20websites%20use%20an%20API%20(Application%20Programming%20Interface)%20in,to%20exchange%20data%20and%20requests.

• One ore two APIs? are there one or two APIs, one on the server side and one on the client side, or just one API? Does it depend on the API? Server APIs expose endpoints....How would a client-API work? For example, are Google Maps APIs client-APIs or server-APIs?
• Python script: the short python script in the attached figure represents an API call to the yelp server...An API has a URL (URI) that specifies both the IP address the API endpoint, and possibly other parameters. When working with the requests library, we can use the HTTP verbs like GET, PUT, DELETE, etc. Here we use GET to get some information back as a json file. We are essentially using a python library, requests, and its functions get to perform the communication. But this code snippet is NOT really the API itself...even if we described an API as a simple set of rules implemented by the functions of the library requests, correct?

Analogy: finally, we know that an IP address identifies a specific host on the Internet and the port number identifies the specific service offered by the host. For example, port 80 is the door that a client knocks at to get web-like information from a server....The analogy is that the IP address identifies the house and the port identifies one of the house access doors to different rooms in the house where different resources are stored...Using this oversimplified analogy, what would an API and an API endpoint represent?

THANK YOU!

 

[jack action]

Analogy: finally, we know that an IP address identifies a specific host on the Internet and the port number identifies the specific service offered by the host. For example, port 80 is the door that a client knocks at to get web-like information from a server....The analogy is that the IP address identifies the house and the port identifies one of the house access doors to different rooms in the house where different resources are stored...Using this oversimplified analogy, what would an API and an API endpoint represent?

API is the language (English, German, Japanese) chosen to communicate between the client and the resource. What are the endpoints of the Japanese language? The one who talks and the ones who listen (or the one who writes and the ones who read), so the client and resource themselves.

The API specification would be the dictionary and grammar of the chosen language.

With Web API, we mean any URL that doesn't return an HTML document or a file stored on the server. But technically, HTTP and HTML are some sort of APIs.

 

[fog37]

API is the language (English, German, Japanese) chosen to communicate between the client and the resource. What are the endpoints of the Japanese language? The one who talks and the ones who listen (or the one who writes and the ones who read), so the client and resource themselves.

The API specification would be the dictionary and grammar of the chosen language.

With Web API, we mean any URL that doesn't return an HTML document or a file stored on the server. But technically, HTTP and HTML are some sort of APIs.

Hello again. I did some more reading....It seems to me that a public, server-side web API is really like an intermediary program/software that contains functions that can be used by a client program to interact with and ask for specific resources to a remote web server (the resources are truly located on a database connected to the web server). For example, in Python, we can write a script containing a bunch of function definitions to use/import later in other scripts. So I am thinking about an API as a set of functions used by the client. These simpler to use functions essentially wrap more complicated actions executed by the server. By example, let's think about the libraries keras and Tensorflow TF: Keras is an API for TF since it has functions to make TF do complicated things. Keras' functions indirectly use TF functions that would be harder to use for....

So there are really 3 programs: the web client program (an app we develop, a local python script, etc), the remote web server program, and the API program (the collected API functions must constitute a program) which is saved/ocated on the same physical machine as the web server program.

What if instead of asking a web API for structured data, which would be sent back to the client as json or XML file, the client was requesting unstructured data like images or audio files? How would the data be sent back? REST APIs provide the requested content in the form of json files...What data format would be used for unstructured data?

In regards to API's URLs: APIs have URLs just like regular webpages. the client program, like the browser, uses HTTP and the API's URL to use the API. Lots of things on common. The main difference seems to be the type of data sent back to the client: markup document in the case of a webpage. But a browser could also request a pdf file, which not markup language...There is not API involved there, correct?

URL are just IP addresses. Therefore an API has an IP address. Is the API's IP address the same IP for the remote web server? Do APIs also have ports? I don't think so. API simply translate the client's request to the web server...somewhere in there, the correct port must be specified based on the requested information...

 

[PeterDonis]

the remote web server program, and the API program

For APIs that run on servers, the "remote web server program" usually is "the API program". There aren't two separate programs, one that runs the "web server" and one that runs the "API". The program that serves the API is the web server. The API is just a particular set of web requests that a client can make.

What data format would be used for unstructured data?

Whatever format is appropriate for the particular use case.

You seem to have an idea that there is one way that these things are done and you want to know what it is. That's not the case. There are at least as many ways of doing these things as there are use cases.

a browser could also request a pdf file, which not markup language...There is not API involved there, correct?

It depends on the particular use case.

URL are just IP addresses.

More precisely, they refer to IP addresses; your computer makes a DNS request to get the IP address that corresponds to a URL.

Is the API's IP address the same IP for the remote web server?

It depends on the particular use case.

Do APIs also have ports?

It depends on the particular use case.

 

[jack action]

What if instead of asking a web API for structured data, which would be sent back to the client as json or XML file, the client was requesting unstructured data like images or audio files? How would the data be sent back? REST APIs provide the requested content in the form of json files...What data format would be used for unstructured data?

Files are structured data in some way. You are always sending a bunch of 0 and 1, even when sending text files. The structure is to know how to decode them. An image can easily be sent as a base64 encoded string.

But if you want to send an image or an audio file in JSON or XML format, you usually send URLs where you can find those files.

URL are just IP addresses.

No. Domain names represent IP addresses.

A URL contains a host - a domain name or even the IP address itself - and many other information, like the scheme, the port, the path, the query, etc. The URL http://172.217.13.206 in your favorite browser will send you to google.com. (There will be a security warning from your browser because there are no valid certificates for that address, but if you ignore it you will be redirected to https://www.google.com.) Actually, google.com sends you to 172.217.13.206 ...

A URL - Uniform Resource Locator - points to a resource. Pretty much any resource is technically an API: a program uses the URL to fetch data and, based on the API specification, decode it and does something with it.

 

[txxnano]

What I love about API is that the sole definition of their name defines what they are. Application Programming Interface, ideally, should not be restricted by servers, technology, or even machines; it should be a contract you choose to use against a pair to communicate about a topic.

You can decide to use a server because it is faster to convey nowadays to a standard using HTTP, but you can also (and it's already being in used) in your local machine thousands of calls to standard APIs that are not servers (malloc(3): https://man7.org/linux/man-pages/man3/malloc.3.html)

Ideally, if you are a software developer, you (thankfully) will need to deal with mostly how servers communicate at a high-level abstraction and never to the point of understanding if you should or should not use a server.

 

[WWGD]

Look up too, ODBC, which is an abstraction layer between data sources, which decides the needed api, drivers, depending on data source and destination, handshake /inter-process communication.