You are here:

C++/Sorting logic


I have file having numbers written in one column and the size of the file is 1GB.But in my machine memory is of 10 MB.
How to sort all the data from the file and put it in separate file?

Hello Plaban

In general, the way to do it is to read in your original file in small parts and sort each part. If you can have only 10 MB in memory at one time, then you can sort only 10 MB at a time. Write each part out to its own file on disk. At this point you will have a collection of files of about 10MB in size each. This is your first generation of files. Now take this generation of files, two at a time and merge their contents, to produce the next generation of files. Repeat this merging until the final generation consists of 1 file, 1GB long, containing all the numbers in sorted order.

Your program has 2 phases. The first phase is the reading of the original file in segments, sorting the segments, and creating the first generation of files.

The second phase is the merging of generation N to get generation N+1.

You should come up with a naming scheme to easily identify the file M in generation N. For example if your original file is called mydata.dat, then the second file in the first generation could be called mydata_1_2.dat. You can delete each file once you are done with it.

That is the general plan. Start with the first phase. If you have trouble with it, show me what you have attempted or tell me what part you don't understand and I will help you more.

Kindest regards


All Answers

Answers by Expert:

Ask Experts




No longer taking questions.


No longer taking questions.

No longer taking questions.

©2017 All rights reserved.