Your browser does not have javaScript enabled. Please enable it, or resize your browsers width so you can view the scriptless navigation bar below and the page's content.
Using NCO with CCR netCDF Files
NCO Tutorial
Other NCO Documentation
What is NCO and why should I use it?
NCO is a group of programs
that allows you to manipulate netCDF files. You can edit file attributes,
rename variables, extract variables for certain dimensions (such as
a given time slice or geographical area), and create ensembles and averages.
You can do much of the same things with NCL, but NCO is much easier
to use. The commands are given at the unix command prompt (in our examples,
plum-%), or they can be strung together in a shell script, depending
on the amount of manipulation that needs to be done. The official documentation
for NCO can be found here.
We've put together some examples here using some CCR files. In all cases,
the original data file is available for download.
How to use NCO and these
examples: first, make a back-up copy of any files you wish to process.
NCO will usually give you an error message if something goes wrong,
but it's always best to have a copy of your unaltered data. On this
page, all commands are shown in red type.
On dave (dave.ucar.edu) at NCAR, you should use a newer version
of NCO. The path is /usr/local/apps/nco-2.7.1/bin/
NCEA
Ncea creates ensemble averages.
It is generally used when you have several years of data, and you want
to create an ensemble, or average, of those years (i.e., going from
individual year files to mone files).
Example: you have five files,
each of which have twelve months of data in them (Jan-Dec). You want
to create an ensemble (average) of these five years.
If the files are: (click on them to download):
ha.F1.PBC_TPBOX.00510.nc
ha.F1.PBC_TPBOX.00511.nc
ha.F1.PBC_TPBOX.00512.nc
ha.F1.PBC_TPBOX.00513.nc
ha.F1.PBC_TPBOX.00514.nc
Then the command would be:
plum-% ncea
-n 5,5,1 ha.F1.PBC_TPBOX.00510.nc ha.F1.PBC_TPBOX.510_514.nc
where
- 5 is the number of files in the ensemble
- 5 is the digit_number, the fixed number of numeric digits comprising the numeric_suffix (00510, 00511, 00512, etc.)
- 1 is the numeric_increment, the constant, integer-valued difference between the numeric_suffix of any two consecutive files
- ha.F1.PBC_TPBOX.00510.nc is the first input file in the series
- ha.F1.PBC_TPBOX.510_514.nc is the output file name
NCEA can also be used to create seasonal files from monthly files.
For example, if you have three files representing March (moni03.nc),
April (moni04.nc), and May (moni05.nc), you can create a file that is
the average of those three files - a spring average (March-April-May).
The command would be:
plum-% ncea -n 3,2,1 moni03.nc mami.nc
As before, 3 is the number of files, 2 is the number of digits in the
file name, and 1 is the increment.
NCRCAT
Ncrcat concatenates (glues together) files across the record dimension,
which is usually time. It can be used to create files that contain multiple
years of the same month or season, such as "anni" or "juli" files. It
may also be used to glue individual year files into a single file, thus
creating a "moni" file.
Example: you have five files, each of which have twelve months of data
in them (Jan-Dec). You want to glue them together to create a single file
that will contain all the months for five years - a "moni" file. Therefore
instead of having five files with twelve months each, you will end up
with a file with 60 time steps (5 years of 12 months = 60 months).
Using the files from the previous example, the command would be:
plum-% ncrcat -n 5,5,1 ha.F1.PBC_TPBOX.00510.nc ha.F1.PBC_TPBOX.moni.nc
The '-n 5,5,1' convention is the same as for ncea listed above. If
you're curious, you can download and look at the new file, ha.F1.PBC_TPBOX.moni.nc,
or view
the header information for the file.
Example: you have five
files, each of which have twelve months of data in them (Jan-Dec). You
want to create files that contain 5 years of January, 5 years of February
... 5 years of December (i.e., jani,febi,...deci files). You also want
to create an "anni" file (5 years of annual average). First we create
the individual months files, then the anni file. Using the same files
as the above example, the command to create the january individual year
file (jani-we call it "moni01") is:
plum-%
ncrcat -F -d time,1,,12 -n 5,5,1 ha.F1.PBC_TPBOX.00510.nc moni01.nc
To create the febi ("moni02") file, you would change the increment of
time to 2
plum-%
ncrcat -F -d time,2,,12 -n 5,5,1 ha.F1.PBC_TPBOX.00510.nc moni02.nc
And so on. Once you have made the twelve individual files, (moni01.nc
through moni12.nc), you can create an anni file using the ncea command.
plum-%
ncea -p -n 12,2,1 moni01.nc anni.nc
Notice that the file input command "-n 12,2,1" has changed, since we
now have 12 input files (moni01-mon12), and each file name has two digits
(moni01.nc-moni12.nc).
NCKS
Ncks - or nc kitchen sink
- is probably the most useful of the NCO operators. You can use it to
extract a variable from a file, or to extract multiple variables, or
to get variables at certain times or geographical areas. It's especially
helpful for when you are moving files from NCAR to CCR. You can extract
just the variables you want, making the file sizes much smaller and
more manageable.
Example: extract a single
variable from a netCDF file
plum-% ncks
-v PRECTMM GEN2AGCM-CONB-MONE0509.nc GEN2AGCM-CONB-MONE0509.prectmm.nc
Example: extract multiple
variables from a netCDF file
plum%
ncks -v PRECTMM,PSLEVMB,CLOUD GEN2AGCM-CONB-MONE0509.nc GEN2AGCM-CONB-SEAE0509.3vars.nc
Example: extract a single
variable from a netCDF file for the month of June from a mone file.
Note: NCO counts from
zero. To count from one, use the -F flag in any operation.
plum-%
ncks -d time,5 -v PRECTMM GEN2AGCM-CONB-MONE0509.nc GEN2AGCM-CONB-MONE0509.prectmm.may.nc
You would get the same result
using the following:
plum-% ncks
-F -d time,6 -v PRECTMM GEN2AGCM-CONB-MONE0509.nc GEN2AGCM-CONB-MONE0509.prectmm.may.nc
NCDIFF
Ncdiff allows you to two
subtract one netCDF file from another and create a new file with the
resulting deltas.
Example: Create a difference
file from two yearly FOAM files.
plum-%
ncdiff ha.F1.PBC_TPBOX.00511.nc ha.F1.PBC_TPBOX.00510.nc ha.F1.PBC_TPBOX.00511-00510.nc
In the example, we are subtracting
the file ha.F1.PBC_TPBOX.00510.nc from the file ha.F1.PBC_TPBOX.00511.nc
and creating a new file that will contain the differences, ha.F1.PBC_TPBOX.00511-00510.nc.
You can download the three
files and examine the output for yourself:
ha.F1.PBC_TPBOX.00510.nc
ha.F1.PBC_TPBOX.00511.nc
ha.F1.PBC_TPBOX.00511-00510.nc
NCRENAME
Ncrename is used to rename
variables in a netCDF file.
Example: In many CCR files,
precipitation-evaporation is named P-E. However, GrADS and many other
software packages will not recognize variables with a minus sign in
them. So in order to look at P-E, it must be renamed.
The following command will
change the name P-E to PME. The file GEN2AGCM-CONB-MONE0509.rename.nc
will be identical to GEN2AGCM-CONB-MONE0509.nc except that P-E will
be named PME. You can download the sample file here: GEN2AGCM-CONB-SEAE0509.nc.
Here is the file metadata/header information (retrieved via ncdump -h):
GEN2AGCM-CONB-SEAE0509.ncdump.txt.
plum-% ncrename
-v P-E,PME GEN2AGCM-CONB-MONE0509.nc GEN2AGCM-CONB-MONE0509.rename.nc
If you wanted to, you could
write over the original file and not create a new file.
plum-% ncrename
-v P-E,PME GEN2AGCM-CONB-MONE0509.nc
Ncrename can also be used
to rename coordinate variables such as latitude and longitude. This
is a little trickier because these latitude and longitude are both dimensions
and variables within a file. Therefore, you have to rename both the
dimension and and the variable.
Example: rename latitude
and longitude to lat and lon, respectively, using the same file as above,
GEN2AGCM-CONB-MONE0509.nc.
plum-% ncrename
-d longitude,lon -d latitude,lat -v longitude,lon -v latitude,lat GEN2AGCM-CONB-MONE0509.nc
GEN2AGCM-CONB-SEAE0509.rename2.nc
