nio

Java Nio Scatter/Gather Example

Photo of Yatin YatinOctober 10th, 2017Last Updated: March 12th, 2019
2 222 6 minutes read

In Java Nio, the channel provides an important capability known as scatter/gather or vectored I/O in some circles. Scatter/gather is a simple yet powerful concept and this tutorial explains how scatter/gather can be really useful in situations where developers need to separate work with the various parts of the transmitted data.
 
 
 
 
 
 
 

Table Of Contents

1. Introduction
1.1 Java Nio Scatter/Gather
2. Java Nio Scatter/Gather Example
2.1 Tools Used
2.2 Project Structure
2.3 Project Creation
3. Application Building
3.1 Java Class Creation
4. Run the Application
5. Project Demo
6. Conclusion
7. Download the Eclipse Project

1. Introduction

Java Nio was developed to allow the Java programmers implement the high-speed input-output operations without using the custom native code. Nio moves the time-taking I/O activities like filling, namely and draining buffers etc back into the operating system, thus allows the great increase in the operational speed.

Java Nio consists of the following core components:

  • Channel & Buffers: In standard I/O API the character streams and the byte streams are used but in NIO, developers work with the channels and buffers. In this case, the data is always written from a buffer to a channel and read from a channel to a buffer

    Fig. 1: Channel & Buffers
    Fig. 1: Channel & Buffers
  • Selectors: It is an object that can be used for monitoring the multiple channels for events like data arrived, connections opened etc. Thus, a single thread can monitor the multiple channels for the data

    Fig. 2: A Thread uses a Selector to handle 3 Channel's
    Fig. 2: A Thread uses a Selector to handle 3 Channel’s
  • Non-blocking I/O: Here the application immediately returns the available data and application should have a pooling mechanism to find out when more data is available

Do note, Java NIO has more components and classes but the Channel, Buffer, and Selector are used as the core of the API.

1.1 Java Nio Scatter/Gather

Java Nio comes with built-in scatter/gather support. Scatter/gather are the concepts used for reading from, and writing to channels i.e. Scatter/Gather is a technique through which bytes can be read from a stream into a set of buffers with a single read() invocation and bytes can be written from a set of buffers to a stream with a single write() invocation.

Scatter/Gather can be really useful in situations where developers need to separate work with the various parts of the transmitted data. For e.g., if a message consists of a header and a body, developers can keep the header and body in separate buffers for easy processing.

1.1.1 Scattering Reads

The ‘scattering read‘ is used for reading the data from a single channel into the multiple buffers. Here is an illustration of the Scatter principle:

Fig. 3: Java Nio Scattering Reads
Fig. 3: Java Nio Scattering Reads

Let’s see a code snippet that performs a scattering read operation.

ByteBuffer bblen1 = ByteBuffer.allocate(1024);
ByteBuffer bblen2 = ByteBuffer.allocate(1024);

scatter.read(new ByteBuffer[] {bblen1, bblen2});

Notice how the buffers are first inserted into an array, then the array is passed as a parameter to the scatter.read() method. The read() method writes the data from the channel in a sequence that buffers occur in an array. Once a buffer is full, the channel moves on to fill the next buffer.

1.1.2 Gathering Writes

The ‘gathering write‘ is used for writing the data from the multiple buffers into a single channel. Here is an illustration of the Gather principle:

Fig. 4: Java Nio Gathering Writes
Fig. 4: Java Nio Gathering Writes

Let’s see a code snippet that performs a gathering write operation.

ByteBuffer bblen1 = ByteBuffer.allocate(1024);
ByteBuffer bblen2 = ByteBuffer.allocate(1024);

gather.write(new ByteBuffer[] {bblen1, bblen2});

The array of buffers are passed into the write() method, which writes the content of the buffers in a sequence they are encountered in an array. Do note, only the data between the position and the limit of the buffers are written.

Now, open up the Eclipse IDE and let’s write a quick example to understand how to use this feature!

2. Java Nio Scatter/Gather Example

2.1 Tools Used

We are using Eclipse Kepler SR2, JDK 8 and Maven. Having said that, we have tested the code against JDK 1.7 and it works well.

2.2 Project Structure

Firstly, let’s review the final project structure, in case you are confused about where you should create the corresponding files or folder later!

Fig. 5: Java Nio Scatter/Gather Project Structure
Fig. 5: Java Nio Scatter/Gather Project Structure

2.3 Project Creation

This section will demonstrate on how to create a Java-based Maven project with Eclipse. In Eclipse IDE, go to File -> New -> Maven Project.

Fig. 6: Create Maven Project
Fig. 6: Create Maven Project

In the New Maven Project window, it will ask you to select project location. By default, ‘Use default workspace location’ will be selected. Select the ‘Create a simple project (skip archetype selection)’ checkbox and just click on next button to proceed.

Fig. 7: Project Details
Fig. 7: Project Details

It will ask you to ‘Enter the group and the artifact id for the project’. We will input the details as shown in the below image. The version number will be by default: 0.0.1-SNAPSHOT.

Fig. 8: Archetype Parameters
Fig. 8: Archetype Parameters

Click on Finish and the creation of a maven project is completed. If you observe, it has downloaded the maven dependencies and a pom.xml file will be created. It will have the following code:

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>JavaNioScatterGather</groupId>
	<artifactId>JavaNioScatterGather</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>
</project>

Developers can start adding the dependencies that they want like JUnit etc. Let’s start building the application!

Download NOW!

Once the package is created in the application, we will need to create the implementation class. Right-click on the newly created package: New -> Class.

Fig. 11: Java Class Creation
Fig. 11: Java Class Creation

A new pop window will open and enter the file name as: GatheringAndScattering. The receiver class will be created inside the package: com.jcg.java.nio.

Fig. 12: Java Class (GatheringAndScattering.java)
Fig. 12: Java Class (GatheringAndScattering.java)

3.1.1 Implementation of Utility Class

Let’s see the simple example of two buffers where both buffers hold the data and developers want to write using the scatter/gather mechanism. Add the following code to it:

GatheringAndScattering.java

package com.jcg.java.nio;

import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.GatheringByteChannel;
import java.nio.channels.ScatteringByteChannel;
import java.nio.charset.Charset;

public class GatheringAndScattering {

	private static String FILENAME = "config/sample.txt";
	private static Charset charset = Charset.forName("UTF-8");

	public static void main(String[] args) {

		String data1 = "1A channel that can write bytes from a sequence of buffers.";
		String data2 = "23A channel that can read bytes into a sequence of buffers.";

		// We are going to store 2 data's to the file using GatheringByteChannel
		gathering(data1, data2);

		scattering();
	}

	private static void scattering() {
		ByteBuffer bblen1 = ByteBuffer.allocate(1024);
		ByteBuffer bblen2 = ByteBuffer.allocate(1024);

		ByteBuffer bbdata1 = null;
		ByteBuffer bbdata2 = null;

		FileInputStream in;
		try {
			in = new FileInputStream(FILENAME);
			ScatteringByteChannel scatter = in.getChannel();

			// Read 2 length first to get the length of 2 data
			scatter.read(new ByteBuffer[] {bblen1, bblen2});

			// We have to call rewind if want to read buffer again. It is same as bblen1.position(0).
			// bblen1.rewind();
			// bblen2.rewind();

			// Seek position to 0 so that we can read the data again.
			bblen1.position(0);
			bblen2.position(0);

			int len1 = bblen1.asIntBuffer().get();
			int len2 = bblen2.asIntBuffer().get();

			// Try to test lengths are correct or not.
			System.out.println("Scattering : Len1= " + len1);
			System.out.println("Scattering : Len2= " + len2);

			bbdata1 = ByteBuffer.allocate(len1);
			bbdata2 = ByteBuffer.allocate(len2);

			// Read data from the channel
			scatter.read(new ByteBuffer[] {bbdata1, bbdata2});

		} catch (FileNotFoundException exObj) {
			exObj.printStackTrace();
		} catch (IOException ioObj) {
			ioObj.printStackTrace();
		}

		// Testing the data is correct or not.
		String data1 = new String(bbdata1.array(), charset);
		String data2 = new String(bbdata2.array(), charset);

		System.out.println(data1);
		System.out.println(data2);
	}

	private static void gathering(String data1, String data2) {
		// Store the length of 2 data using GatheringByteChannel
		ByteBuffer bblen1 = ByteBuffer.allocate(1024);
		ByteBuffer bblen2 = ByteBuffer.allocate(1024);

		// Next two buffer hold the data we want to write
		ByteBuffer bbdata1 = ByteBuffer.wrap(data1.getBytes());
		ByteBuffer bbdata2 = ByteBuffer.wrap(data2.getBytes());

		int len1 = data1.length();
		int len2 = data2.length();

		// Writing length(data) to the Buffer
		bblen1.asIntBuffer().put(len1);
		bblen2.asIntBuffer().put(len2);

		System.out.println("Gathering : Len1= " + len1);
		System.out.println("Gathering : Len2= " + len2);

		// Write data to the file
		try { 
			FileOutputStream out = new FileOutputStream(FILENAME);
			GatheringByteChannel gather = out.getChannel();						
			gather.write(new ByteBuffer[] {bblen1, bblen2, bbdata1, bbdata2});
			out.close();
			gather.close();
		} catch (FileNotFoundException exObj) {
			exObj.printStackTrace();
		} catch(IOException ioObj) {
			ioObj.printStackTrace();
		}
	}
}

4. Run the Application

To run the Java Nio application, Right-click on the GatheringAndScattering class -> Run As -> Java Application. Developers can debug the example and see what happens after every step!

Fig. 13: Run Application
Fig. 13: Run Application

5. Project Demo

When developers run the above program, the code shows the following status as output.

Fig. 14: Console Output
Fig. 14: Console Output

That’s all for this post. Happy Learning!!

6. Conclusion

Scatter and Gather are powerful tools when they are used properly and they allow developers to delegate the grunt work to an operating system. The operating system separates out the data into multiple buckets or assembles the disparate data chunks into a whole. This can be a huge win because the operating system is highly optimized for this sort of thing and saves the work of moving things around; thus avoiding buffer copies and reducing the number of code that developers need to write and debug. Since developers are basically assembling the data by providing references to the data containers, the various chunks can be assembled in different ways by building the multiple arrays of the buffers referenced in different combinations.

That’s all for this tutorial and I hope this article served you whatever you were looking for.

7. Download the Eclipse Project

This was an example of Java Nio for the beginners.

Download
You can download the full source code of this example here: Sign up
Tags
Photo of Yatin YatinOctober 10th, 2017Last Updated: March 12th, 2019
2 222 6 minutes read
Photo of Yatin

Yatin

An experience full-stack engineer well versed with Core Java, Spring/Springboot, MVC, Security, AOP, Frontend (Angular & React), and cloud technologies (such as AWS, GCP, Jenkins, Docker, K8).
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

2 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Jarrod Roberson
Jarrod Roberson
8 years ago

This is great and all but you did not provide an actual practical example of the one use case you mentioned “header and body in separate buffers”. This tutorial is pretty useless without an actual practical example. It is just more noise on the internet to take up your time and not learn anything new.

0
Reply
Yatin
Author
Yatin
8 years ago

Jarrod,

Hope you are doing well. Thanks for pointing out the use case but due to the unavailability of the FTP resource at that moment I wasn’t able to write down an actual practical code (i.e. clearing showing the header & body separation) and hence had to redirect the code approach from FTP to File. I will follow up with the support team and would request them for one more article on this topic!

Have a great day!

0
Reply
WhatsApp
Back to top button