Package com.codename1.io

package com.codename1.io

Networking, Storage, Filesystem & related API's

The IO package includes all of the main features related to storage and networking with the exception of SQL & XML parsing.

Storage

com.codename1.io.Storage is accessed via the Storage class. It is a flat filesystem like interface and contains the ability to list/delete and write to named storage entries.

The com.codename1.io.Storage API also provides convenient methods to write objects to com.codename1.io.Storage and read them from com.codename1.io.Storage specifically readObject & writeObject.

Notice that objects in com.codename1.io.Storage are deleted when an app is uninstalled but are retained between application updates.

The sample code below demonstrates listing the content of the storage, adding/viewing and deleting entries within the storage:

public void pictureUpload(final Callback resultURL) {
    String picture = Capture.capturePhoto(1024, -1);
    if(picture!=null){
        String filestack = "https://www.filestackapi.com/api/store/S3?key=MY_KEY&filename=myPicture.jpg";
        MultipartRequest request = new MultipartRequest() {
           protected void readResponse(InputStream input) throws IOException  {
              JSONParser jp = new JSONParser();
              Map result = jp.parseJSON(new InputStreamReader(input, "UTF-8"));
              String url = (String)result.get("url");
              if(url == null) {
                 resultURL.onError(null, null, 1, result.toString());
                 return;
              }
              resultURL.onSucess(url);
           }
        };
        request.setUrl(filestack);
        try {
            request.addData("fileUpload", picture, "image/jpeg");
            request.setFilename("fileUpload", "myPicture.jpg");
            NetworkManager.getInstance().addToQueue(request);
        } catch(IOException err) {
            err.printStackTrace();
        }
    }
}

The Preferences API

com.codename1.io.Storage also offers a very simple API in the form of the com.codename1.io.Preferences class. The com.codename1.io.Preferences class allows developers to store simple variables, strings, numbers, booleans etc. in storage without writing any storage code. This is a common use case within applications e.g. you have a server token that you need to store you can store & read it like this:

// save a token to storage
Preferences.set("token", myToken);

// get the token from storage or null if it isn't there
String token = Preferences.get("token", null);

This gets somewhat confusing with primitive numbers e.g. if you use Preferences.set("primitiveLongValue", myLongNumber) then invoke Preferences.get("primitiveLongValue", 0) you might get an exception!

This would happen because the value is physically a Long object but you are trying to get an Integer. The workaround is to remain consistent and use code like this Preferences.get("primitiveLongValue", (long)0).

File System

com.codename1.io.FileSystemStorage provides file system access. It maps to the underlying OS's file system API providing most of the common operations expected from a file API somewhat in the vain of java.io.File & java.io.FileInputStream e.g. opening, renaming, deleting etc.

Notice that the file system API is somewhat platform specific in its behavior. All paths used the API should be absolute otherwise they are not guaranteed to work.

The main reason java.io.File & java.io.FileInputStream weren't supported directly has a lot to do with the richness of those two API's. They effectively allow saving a file anywhere, however mobile devices are far more restrictive and don't allow apps to see/modify files that are owned by other apps.

File Paths & App Home

All paths in com.codename1.io.FileSystemStorage are absolute, this simplifies the issue of portability significantly since the concept of relativity and current working directory aren't very portable.

All URL's use the / as their path separator we try to enforce this behavior even in Windows.

Directories end with the / character and thus can be easily distinguished by their name.

The com.codename1.io.FileSystemStorage API provides a getRoots() call to list the root directories of the file system (you can then "dig in" via the listFiles API). However, this is confusing and unintuitive for developers.

To simplify the process of creating/reading files we added the getAppHomePath() method. This method allows us to obtain the path to a directory where files can be stored/read.

We can use this directory to place an image to share as we did in the share sample.

Warning: A common Android hack is to write files to the SDCard storage to share them among apps. Android 4.x disabled the ability to write to arbitrary directories on the SDCard even when the appropriate permission was requested.

A more advanced usage of the com.codename1.io.FileSystemStorage API can be a com.codename1.io.FileSystemStorage Tree:

Form hi = new Form("FileSystemTree", new BorderLayout());
TreeModel tm = new TreeModel() {
@Override
    public Vector getChildren(Object parent) {
        String[] files;
        if(parent == null) {
            files = FileSystemStorage.getInstance().getRoots();
            return new Vector(Arrays.asList(files));
        } else {
            try {
                files = FileSystemStorage.getInstance().listFiles((String)parent);
            } catch(IOException err) {
                Log.e(err);
                files = new String[0];
            }
        }
        String p = (String)parent;
        Vector result = new Vector();
        for(String s : files) {
            result.add(p + s);
        }
        return result;
    }
@Override
    public boolean isLeaf(Object node) {
        return !FileSystemStorage.getInstance().isDirectory((String)node);
    }
};
Tree t = new Tree(tm) {
@Override
    protected String childToDisplayLabel(Object child) {
        String n = (String)child;
        int pos = n.lastIndexOf("/");
        if(pos

Storage vs. File System

    The question of storage vs. file system is often confusing for novice mobile developers. This embeds
    two separate questions:


-


Why are there 2 API's where one would have worked?



-


Which one should I pick?



    The main reasons for the 2 API's are technical. Many OS's provide 2 ways of accessing data
    specific to the app and this is reflected within the API. E.g. on Android the
    `com.codename1.io.FileSystemStorage` maps to API's such as `java.io.FileInputStream`
    whereas the `com.codename1.io.Storage` maps to `Context.openFileInput()`.

    The secondary reason for the two API's is conceptual. `com.codename1.io.FileSystemStorage` is
    more powerful and in a sense provides more ways to fail, this is compounded by the complex
    on-device behavior of the API. `com.codename1.io.Storage` is designed to be friendlier to the
    uninitiated and more portable.

    You should pick `com.codename1.io.Storage` unless you have a specific requirement that prevents it.
    Some API's such as `Capture` expect a `com.codename1.io.FileSystemStorage` URI
    so in those cases this would also be a requirement.

    Another case where `com.codename1.io.FileSystemStorage` is beneficial is the case of hierarchy or
    native API usage. If you need a a directory structure or need to communicate with a native
    API the `com.codename1.io.FileSystemStorage` approach is usually easier.

    **Warning: **In some OS's the `com.codename1.io.FileSystemStorage` API can find the
    content of the `com.codename1.io.Storage` API. As one is implemented on top of the other. This is
    undocumented behavior that can change at any moment!

Network Manager & Connection Request

    One of the more common problems in Network programming is spawning a new thread to handle
    the network operations. In Codename One this is done seamlessly and becomes unessential
    thanks to the `com.codename1.io.NetworkManager`.

    `com.codename1.io.NetworkManager` effectively alleviates the need for managing network threads by
    managing the complexity of network threading. The connection request class can be used to
    facilitate web service requests when coupled with the JSON/XML parsing capabilities.

    To open a connection one needs to use a `com.codename1.io.ConnectionRequest`
    object, which has some similarities to the networking mechanism in JavaScript but is obviously somewhat more
    elaborate.

You can send a get request to a URL using something like:

```java
ConnectionRequest request = new ConnectionRequest(url, false);
request.addResponseListener((e) -> {
// process the response
});

// request will be handled asynchronously
NetworkManager.getInstance().addToQueue(request);

Notice that you can also implement the same thing and much more by avoiding the response
listener code and instead overriding the methods of the `com.codename1.io.ConnectionRequest` class
which offers multiple points to override e.g.

ConnectionRequest request = new ConnectionRequest(url, false) {
protected void readResponse(InputStream input) {
// just read from the response input stream
}

protected void postResponse() {
// invoked on the EDT after processing is complete to allow the networking code
// to update the UI
}

protected void buildRequestBody(OutputStream os) {
// writes post data, by default this "just works" but if you want to write this
// manually then override this
}
};
NetworkManager.getInstance().addToQueue(request);

Notice that overriding `buildRequestBody(OutputStream)` will only work for
`POST` requests and will replace writing the arguments.

**Important:** You don't need to close the output/input streams passed to the
request methods. They are implicitly cleaned up.

`com.codename1.io.NetworkManager` also supports synchronous requests which work in a similar
way to `Dialog` via the `invokeAndBlock` call and thus don't block
the EDT illegally. E.g. you can do something like this:

ConnectionRequest request = new ConnectionRequest(url, false);
// request will be handled synchronously
NetworkManager.getInstance().addToQueueAndWait(request);
byte[] resultOfRequest = request.getData();

Notice that in this case the `addToQueueAndWait` method returned after the
connection completed. Also notice that this was totally legal to do on the EDT!

Threading

By default the `com.codename1.io.NetworkManager` launches with a single network thread. This is
sufficient for very simple applications that don't do too much networking but if you need to
fetch many images concurrently and perform web services in parallel this might be an issue.

**Warning:** Once you increase the thread count there is no guarantee of order for your requests.
Requests
might not execute in the order with which you added them to the queue!

To update the number of threads use:

NetworkManager.getInstance().updateThreadCount(4);

All the callbacks in the `ConnectionRequest` occur on the network thread and
**not on the EDT**!

There is one exception to this rule which is the `postResponse()` method designed
to update the UI after the networking code completes.

**Important:** Never change the UI from a `com.codename1.io.ConnectionRequest`
callback. You can either use a listener on the `com.codename1.io.ConnectionRequest`, use
`postResponse()` (which is the only exception to this rule) or wrap your UI code with
`com.codename1.ui.Display#callSerially(java.lang.Runnable)`.

Arguments, Headers & Methods

HTTP/S is a complex protocol that expects complex encoded data for its requests. Codename
One tries to simplify and abstract most of these complexities behind common sense API's while
still providing the full low level access you would expect from such an API.

Arguments

HTTP supports several "request methods", most commonly `GET` &
`POST` but also a few others such as `HEAD`, `PUT`,
`DELETE` etc.

Arguments in HTTP are passed differently between GET and POST methods. That is what the setPost method in Codename One determines, whether arguments added to the request should be placed using the GET semantics or the POST semantics.

So if we continue our example from above we can do something like this:

ConnectionRequest request = new ConnectionRequest(url, false);
request.addArgument("MyArgName", value);

This will implicitly add a get argument with the content of `value`. Notice that we
don't really care what value is. It's implicitly HTTP encoded based on the get/post semantics.
In this case it will use the get encoding since we passed `false` to the constructor.

A simpler implementation could do something like this:

ConnectionRequest request = new ConnectionRequest(url +
"MyArgName=" + Util.encodeUrl(value), false);

This would be almost identical but doesn't provide the convenience for switching back and
forth between `GET`/`POST` and it isn't as fluent.

We can skip the encoding in complex cases where server code expects illegal HTTP characters (this happens) using the addArgumentNoEncoding method. We can also add multiple arguments with the same key using addArgumentArray.

Methods

As we explained above, the `setPost()` method allows us to manipulate the
get/post semantics of a request. This implicitly changes the `POST`
or `GET` method submitted to the server.

However, if you wish to have finer grained control over the submission process e.g. for making a
`HEAD` request you can do this with code like:

ConnectionRequest request = new ConnectionRequest(url, false);
request.setHttpMethod("HEAD");

Headers

When communicating with HTTP servers we often pass data within headers mostly for
authentication/authorization but also to convey various properties.

Some headers are builtin as direct API's e.g. content type is directly exposed within the API
since it's a pretty common use case. We can set the content type of a post request using:

ConnectionRequest request = new ConnectionRequest(url, true);
request.setContentType("text/xml");

We can also add any arbitrary header type we want, e.g. a very common use case is basic
authorization where the authorization header includes the Base64 encoded user/password
combination as such:

String authCode = user + ":" + password;
String authHeader = "Basic " + Base64.encode(authCode.getBytes());
request.addRequestHeader("Authorization", authHeader);

This can be quite tedious to do if you want all requests from your app to use this header. For this use case you can just use:

String authCode = user + ":" + password;
String authHeader = "Basic " + Base64.encode(authCode.getBytes());
NetworkManager.getInstance().addDefaultHeader("Authorization", authHeader);

Server Headers

Server returned headers are a bit trickier to read. We need to subclass the connection request
and override the `readHeaders` method e.g.:

ConnectionRequest request = new ConnectionRequest(url, false) {
protected void readHeaders(Object connection) throws IOException {
String[] headerNames = getHeaderFieldNames(connection);
for(String headerName : headerNames) {
String headerValue = getHeader(headerName);
//....
}
}
protected void readResponse(InputStream input) {
// just read from the response input stream
}
};
NetworkManager.getInstance().addToQueue(request);

Here we can extract the headers one by one to handle complex headers such as cookies,
authentication etc.

Error Handling

As you noticed above practically all of the methods in the `ConectionRequest`
throw `IOException`. This allows you to avoid the `try`/`catch`
semantics and just let the error propagate up the chain so it can be handled uniformly by
the application.

There are two distinct placed where you can handle a networking error:

The com.codename1.io.ConnectionRequest - by overriding callback methods

The com.codename1.io.NetworkManager error handler

Notice that the `com.codename1.io.NetworkManager` error handler takes precedence thus allowing
you to define a global policy for network error handling by consuming errors.

E.g. if I would like to block all network errors from showing anything to the user I could do
something like this:

NetworkManager.getInstance().addToQueue(request);
NetworkManager.getInstance().addErrorListener((e) -> e.consume());

The error listener is invoked first with the `com.codename1.io.NetworkEvent` matching the
error. Consuming the event prevents it from propagating further down the chain into the
`com.codename1.io.ConnectionRequest` callbacks.

We can also override the error callbacks of the various types in the request e.g. in the case of a
server error code we can do:

ConnectionRequest request = new ConnectionRequest(url, false) {
protected void handleErrorResponseCode(int code, String message) {
if(code == 444) {
// do something
}
}
protected void readResponse(InputStream input) {
// just read from the response input stream
}
};
NetworkManager.getInstance().addToQueue(request);

**Important:** The error callback callback is triggered in the network thread!

As a result it can't access the UI to show a `Dialog` or anything like that.

Another approach is to use the `setFailSilently(true)` method on the
`com.codename1.io.ConnectionRequest`. This will prevent the
`com.codename1.io.ConnectionRequest` from displaying any errors to the user. It's a very
powerful strategy if you use the synchronous version of the API's e.g.:

ConnectionRequest request = new ConnectionRequest(url, false);
request.setFailSilently(true);
NetworkManager.getInstance().addToQueueAndWait(request);
if(request.getResponseCode() != 200) {
// probably an error...
}

This code will only work with the synchronous "AndWait" version of the method since the response
code will take a while to return for the non-wait version.

Error Stream

When we get an error code that isn't 200/300 we ignore the result. This is problematic as the
result might contain information we need. E.g. many webservices provide further XML/JSON
based details describing the reason for the error code.

Calling `setReadResponseForErrors(true)` will trigger a mode where even errors
will receive the `readResponse` callback with the error stream. This also means
that API's like `getData` and the listener API's will also work correctly in
case of error.

GZIP

Gzip is a very common compression format based on the lz algorithm, it's used by web servers
around the world to compress data.

Codename One supports `com.codename1.io.gzip.GZIPInputStream` and
`com.codename1.io.gzip.GZIPOutputStream`, which allow you to compress data
seamlessly into a stream and extract compressed data from a stream. This is very useful and
can be applied to every arbitrary stream.

Codename One also features a `com.codename1.io.gzip.GZConnectionRequest`, which
will automatically unzip an HTTP response if it is indeed gzipped. Notice that some devices (iOS)
always request gzip'ed data and always decompress it for us, however in the case of iOS it
doesn't remove the gziped header. The `GZConnectionRequest` is aware of such
behaviors so its better to use that when connecting to the network (if applicable).

By default `GZConnectionRequest` doesn't request gzipped data (only unzips it
when its received) but its pretty easy to do so just add the HTTP header
`Accept-Encoding: gzip` e.g.:

GZConnectionRequest con = new GZConnectionRequest();
con.addRequestHeader("Accept-Encoding", "gzip");

Do the rest as usual and you should have smaller responses from the servers.

File Upload

`com.codename1.io.MultipartRequest` tries to simplify the process of uploading a file from
the local device to a remote server.

You can always submit data in the `buildRequestBody` but this is flaky and has
some limitations in terms of devices/size allowed. HTTP standardized file upload capabilities
thru the multipart request protocol, this is implemented by countless servers and is well
documented. Codename One supports this out of the box.

Since we assume most developers reading this will be familiar with Java here is the way to
implement the multipart upload in the servlet API.

// File: MultipartClientSample.java
MultipartRequest request = new MultipartRequest();
request.setUrl(url);
request.addData("myFileName", fullPathToFile, "text/plain")
NetworkManager.getInstance().addToQueue(request);

// File: UploadServlet.java

Related Packages

Package	Description
com.codename1.io.gzip	gzip support based on https://github.com/ymnk/jzlib
com.codename1.io.rest	Simplified REST API
com.codename1.io.services	Default WebService implementations
com.codename1.io.tar	Tar support allowing us to package multiple separate files into a single package which is useful in supporting hierarchies in a portable way.

Class	Description
AccessToken	This class represent an access token.
BufferedInputStream	Based on the buffered input stream from the JDK with some minor tweaks to allow external classes to monitor stream status and progress.
BufferedOutputStream	Based on the buffered output stream from the JDK with some minor tweaks to allow external classes to monitor stream status and progress.
CacheMap	A cache map is essentially a hashtable that indexes entries based on age and is limited to a fixed size.
CharArrayReader	A specialized Reader for reading the contents of a char array.
ConnectionRequest	This class represents a connection object in the form of a request response typically common for HTTP/HTTPS connections.
ConnectionRequest.CachingMode	There are 5 caching modes:
ConnectionRequest.SSLCertificate	Encapsulates an SSL certificate fingerprint.
Cookie	A cookie for an HTTP request
CSVParser	The CSVParser allows importing data into applications quickly from a CSV source.
Data	An interface that can be implemented by any object to allow it to be treated as data.
Data.ByteData	Wraps a byte[] array as a Data object.
Data.FileData	Wraps a File as a Data object.
Data.StorageData	Wraps a Storage object as a Data object.
Data.StringData
Externalizable	Externalizable is similar to the Java SE Externalizable interface this interface.
File	This class provides a similar API to java.io.File making it almost into a "drop in" replacement.
File.FileFilter	Interface for filtering files.
File.FilenameFilter	Interface to filter filenames.
FileSystemStorage	Unlike networking, the file system storage mostly tries to emulate java.io.File with some simplifications for mobile devices.
IOProgressListener	Callback for IO updates from a buffered input/output stream
JSONParseCallback	The event based parser allows parsing without creating an object tree by receiving callbacks to this class.
JSONParser	Fast and dirty parser for JSON content on the web, it essentially returns a java.util.Map object containing the object fields mapped to their values.
Log	Pluggable logging framework that allows a developer to log into storage using the file connector API.
MalformedURLException	Thrown when attempting to create a URL that doesn't conform to specification.
MultipartRequest	A multipart post request allows a developer to submit large binary data files to the server in a multipart mime post request.
NetworkEvent	Event containing more meta data for network events which may be error events or an update for progress indication code.
NetworkManager	Main entry point for managing the connection requests, this is essentially a threaded queue that makes sure to route all connections via the network thread while sending the callbacks through the Codename One EDT.
Oauth2	This is a utility class that allows Oauth2 authentication This utility uses the Codename One XHTML Component to display the authentication pages.
Oauth2.RefreshTokenRequest
PreferenceListener	Respond to changes to the Preferences
Preferences	Simple map like class to store application and Codename One preference settings in the com.codename1.io.Storage.
Properties	A Properties object is a Hashtable where the keys and values must be Strings.
Socket	Class implementing the socket API
Socket.StopListening	This interface can be invoked to stop listening on a server socket
SocketConnection	Callback for establishment of a socket connection.
Storage	Abstracts the underlying application specific storage system, unlike the com.codename1.io.FileSystemStorage this class is a higher level abstraction.
URL	This class provides a similar API to URL making it almost into a "drop in" replacement.
Util	Various utility methods used for HTTP/IO operations
WebServiceProxyCall	Utility class used by the webservice proxy code to invoke server code
WebServiceProxyCall.WSDefinition	Webservice definition type, allows defining the argument values for a specific WS call