Neural Network Toolbox 6

Design and simulate neural networks
Neural Network Toolbox extends MATLAB with tools for designing, implementing, visualizing, and simulating neural networks. Neural networks are invaluable for applications where formal analysis would be difficult or impossible, such as pattern recognition and nonlinear system identification and control. Neural Network Toolbox software provides comprehensive support for many proven network paradigms, as well as graphical user interfaces (GUIs) that enable you to design and manage your networks. The modular, open, and extensible design of the toolbox simplifies the creation of customized functions and networks.

Key features

UI for creating, training, and simulating neural networks G uick start wizards for fitting, pattern recognition, and clustering Q upport for the most commonly used supervised and S unsupervised network architectures omprehensive set of training and learning functions C ynamic learning networks, including time delay, nonlinear D autoregressive (NARX), layer-recurrent, and custom dynamic imulink blocks for building neural networks and advanced S blocks for control systems applications upport for automatically generating Simulink blocks from S neural network objects odular network representation that enables an unlimited M number of input-setting layers and network interconnections and a graphical view of network architecture reprocessing and postprocessing functions and Simulink blocks P for improving network training and assessing network performance isualization functions and GUI for viewing network V performance and monitoring the training process
Working with Neural Network Toolbox
Like its counterpart in the biological nervous system, a neural network can learn and therefore can be trained to find solutions, recognize patterns, classify data, and forecast future events. The behavior of a neural network is defined by the way its individual computing elements are connected and by the strength of those connections, or weights. The weights are automatically adjusted by training the network according to a specified learning rule until it performs the desired task correctly. Neural Network Toolbox GUIs make it easy to work with neural networks. The Neural Network Fitting Tool is a wizard that leads you through the process of fitting data using neural networks. You can use the tool to import large and complex data sets, quickly create and train networks, and evaluate network performance.
The Neural Network Fitting Tool (top) and a performance plot (bottom). The Neural Network Fitting Tool guides you through the process of fitting data using neural networks, while additional GUIs are available for other common tasks such as pattern recognition and clustering.
Accelerating the pace of engineering and science
A Simulink model that includes the neural network predictive control block and CSTR plant model (top left). Dialogs and panes let you visualize validation data (lower left) and manage the neural network control block (lower right) and your plant identification (upper right).
A second GUI gives you greater ability to customize the network architecture and learning algorithms. Simple graphical representations enable you to visualize and understand network architecture. Additional GUIs are available for other common tasks including pattern recognition, clustering, and network training.
Feedforward networks have one-way connections from input to output layers. They are most commonly used for prediction, pattern recognition, and nonlinear function fitting. Supported feedforward networks include feedforward backpropagation, cascade-forward backpropagation, feedforward input-delay backpropagation, linear, and perceptron networks. Radial basis networks provide an alternative, fast method for designing nonlinear feedforward networks. Supported variations include generalized regression and probabilistic neural networks. Dynamic networks use memory and recurrent feedback connections to recognize spatial and temporal patterns in data. They are commonly used for time-series prediction, nonlinear dynamic system modeling, and control system applications. Prebuilt dynamic networks in the toolbox include focused and distributed time-delay, nonlin-

ear autoregressive (NARX), layer-recurrent, Elman, and Hopfield networks. The toolbox also supports dynamic training of custom networks with arbitrary connections. LVQ is a powerful method for classifying patterns that are not linearly separable. LVQ lets you specify class boundaries and the granularity of classification.

Network Architectures

Neural Network Toolbox supports both supervised and unsupervised networks.

Unsupervised Networks

Unsupervised neural networks are trained by letting the network continually adjust itself to new inputs. They find relationships within data and can automatically define classification schemes. Neural Network Toolbox supports two types of self-organizing, unsupervised networks: competitive layers and self-organizing maps. Competitive layers recognize and group similar input vectors. By using these groups, the network automatically sorts the inputs into categories.

Supervised Networks

Supervised neural networks are trained to produce desired outputs in response to sample inputs, making them particularly well suited to modeling and controlling dynamic systems, classifying noisy data, and predicting future events. Neural Network Toolbox supports four supervised networks: feedforward, radial basis, dynamic, and learning vector quantization (LVQ).
A three-layer neural network converted into Simulink blocks. Neural network simulation blocks for use in Simulink can be automatically generated with the gensim command.
Self-organizing maps learn to classify input vectors according to similarity. Unlike competitive layers they also preserve the topology of the input vectors, assigning nearby inputs to nearby categories.
A suite of learning functions, including gradient descent, hebbian learning, LVQ, Widrow-Hoff, and Kohonen, is also provided.

Simulink Support

Training and Learning Functions
Training and learning functions are mathematical procedures used to automatically adjust the networks weights and biases. The training function dictates a global algorithm that affects all the weights and biases of a given network. The learning function can be applied to individual weights and biases within a network. Neural Network Toolbox supports a variety of training algorithms, including several gradient descent methods, conjugate gradient methods, the Levenberg-Marquardt algorithm (LM), and the resilient backpropogation algorithm (Rprop). Algorithms can be accessed from the command line or via a training GUI, which shows a diagram of the network being trained, training algorithm choices, and stopping criteria values as the training progresses.

Neural Network Toolbox provides a set of blocks for building neural networks in Simulink software. These blocks are divided into three libraries: ransfer function blocks, which take a netT input vector and generate a corresponding output vector et input function blocks, which take any N number of weighted input vectors, weight layer output vectors, and bias vectors, and return a net-input vector eight function blocks, which apply a neuW rons weight vector to an input vector (or a layer output vector) to get a weighted input value for a neuron ata preprocessing blocks, which map input D and output data into ranges best suited for the neural network to handle directly.
Alternatively, you can create and train your networks in the MATLAB environment and automatically generate network simulation blocks for use with Simulink. This approach also enables you to view your networks graphically.
Control System Applications
Neural Network Toolbox lets you apply neural networks to the identification and control of nonlinear systems. The toolbox includes descriptions, demonstrations, and Simulink blocks for three popular control applications: model predictive control, feedback linearization, and model reference adaptive control. You can incorporate neural network predictive control blocks included in the toolbox into your Simulink models. By changing the parameters of these blocks, you can tailor the networks performance to your application.
Preprocessing and Postprocessing Functions
Preprocessing the network inputs and targets improves the efficiency of neural network training. Postprocessing enables detailed
w w w. m a t h w o r k s. c o m
analysis of network performance. Neural Network Toolbox provides preprocessing and postprocessing functions and Simulink blocks that enable you to: educe the dimensions of the input vectors R using principal component analysis erform regression analysis between the P network response and the corresponding targets cale inputs and targets so that they fall in S the range [-1,1] ormalize the mean and standard deviation N of the training set Automated data preprocessing and data division are built into the network creation process.

Improving Generalization

Improving the networks ability to generalize helps prevent overfitting, a common problem in neural network design. Overfitting occurs when a network has memorized the training set but has not learned to generalize to new inputs. Overfitting produces a relatively small error on the training set but a much larger error when new data is presented to the network. Neural Network Toolbox provides two solutions to improve generalization: regularization and early stopping. Regularization modifies the networks performance function (the measure of error that the training process minimizes). By including the sizes of the weights and biases, training produces a network that performs well with the training data and exhibits smoother behavior when presented with new data. Early stopping uses two different data sets: the training set, to update the weights and biases, and the validation set, to stop training when the network begins to overfit the data.

Required Products

MATLAB

Related Products

Control System Toolbox. Design and analyze control systems Fuzzy Logic Toolbox. Design and simulate fuzzy logic systems Optimization Toolbox. Solve standard and large-scale optimization problems Statistics Toolbox. Perform statistical analysis, modeling, and algorithm development System Identification Toolbox. Create linear and nonlinear dynamic models from measured input-output data
Platform and System Requirements
For platform and system requirements, visit www.mathworks.com/products/neuralnet

Resources

visit www.mathworks.com Technical Support www.mathworks.com/support Online User Community www.mathworks.com/matlabcentral Demos www.mathworks.com/demos Training Services www.mathworks.com/training Third-Party Products and Services www.mathworks.com/connections Worldwide CONTACTS www.mathworks.com/contact e-mail info@mathworks.com
2008 by The MathWorks, Inc. MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective holders.

8511v07 03/08

Neural Network Toolbox Release Notes

How to Contact MathWorks

Web Newsgroup www.mathworks.com/contact_TS.html Technical Support
www.mathworks.com comp.soft-sys.matlab suggest@mathworks.com bugs@mathworks.com doc@mathworks.com service@mathworks.com info@mathworks.com
Product enhancement suggestions Bug reports Documentation error reports Order status, license renewals, passcodes Sales, pricing, and general information
508-647-7000 (Phone) 508-647-7001 (Fax) The MathWorks, Inc. 3 Apple Hill Drive Natick, MA 01760-2098
For contact information about worldwide offices, see the MathWorks Web site. Neural Network Toolbox Release Notes COPYRIGHT 20052011 by The MathWorks, Inc.
The software described in this document is furnished under a license agreement. The software may be used or copied only under the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form without prior written consent from The MathWorks, Inc. FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through the federal government of the United States. By accepting delivery of the Program or Documentation, the government hereby agrees that this software or documentation qualifies as commercial computer software or commercial computer software documentation as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014. Accordingly, the terms and conditions of this Agreement and only those rights specified in this Agreement, shall pertain to and govern the use, modification, reproduction, release, performance, display, and disclosure of the Program and Documentation by the federal government (or other entity acquiring for or through the federal government) and shall supersede any conflicting contractual terms or conditions. If this License fails to meet the governments needs or is inconsistent in any respect with federal procurement law, the government agrees to return the Program and Documentation, unused, to The MathWorks, Inc.

Trademarks

MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective holders.

Patents

MathWorks products are protected by one or more U.S. patents. Please see www.mathworks.com/patents for more information.

Contents

Summary by Version. Version 7.0.1 (R2011a) Neural Network Toolbox Software. Version 7.0 (R2010b) Neural Network Toolbox Software. Version 6.0.4 (R2010a) Neural Network Toolbox Software. Version 6.0.3 (R2009b) Neural Network Toolbox Software. Version 6.0.2 (R2009a) Neural Network Toolbox Software. Version 6.0.1 (R2008b) Neural Network Toolbox Software. Version 6.0 (R2008a) Neural Network Toolbox Software. Version 5.1 (R2007b) Neural Network Toolbox Software. Version 5.0.2 (R2007a) Neural Network Toolbox Software. Version 5.0.1 (R2006b) Neural Network Toolbox Software. Version 5.0 (R2006a) Neural Network Toolbox Software. 1
Version 4.0.6 (R14SP3) Neural Network Toolbox Software. Compatibility Summary for Neural Network Toolbox Software.

Summary by Version

This table provides quick access to whats new in each version. For clarification, see Using Release Notes on page 1. Version (Release) Latest Version V7.0.1 (R2011a) V7.0 (R2010b) V6.0.4 (R2010a) V6.0.3 (R2009b) V6.0.2 (R2009a) V6.0.1 (R2008b) V6.0 (R2008a) V5.1 (R2007b) V5.0.2 (R2007a) V5.0.1 (R2006b) V5.0 (R2006a) V4.0.6 (R14SP3) New Features and Changes No Yes Details No No No No Yes Details Yes Details No No Yes Details No Version Compatibility Considerations No Yes Summary No No No No Yes Summary Yes Summary No No Yes Summary No Fixed Bugs and Known Problems Bug Reports Includes fixes Bug Reports Includes fixes Bug Reports Includes fixes Bug Reports Includes fixes Bug Reports Includes fixes Bug Reports Includes fixes Bug Reports Includes fixes Bug Reports Includes fixes Bug Reports Bug Reports Bug Reports Bug Reports

New features and changes introduced in this version are: New Neural Network Start GUI on page 6 New Time Series GUI and Tools on page 7 New Time Series Validation on page 13 New Time Series Properties on page 13 New Flexible Error Weighting and Performance on page 14 New Real Time Workshop and Improved Simulink Support on page 16 New Documentation Organization and Hyperlinks on page 17 New Derivative Functions and Property on page 18 Improved Network Creation on page 18 Improved GUIs on page 20 Improved Memory Efficiency on page 20 Improved Data Sets on page 20 Updated Argument Lists on page 21
New Neural Network Start GUI
The new nnstart function opens a GUI that provides links to new and existing Neural Network Toolbox GUIs and other resources. The first panel of the GUI opens four "getting started" wizards.
The second panel provides links to other toolbox starting points.
New Time Series GUI and Tools
The new ntstool function opens a wizard GUI that allows time series problems to be solved with three kinds of neural networks: NARX networks (neural auto-regressive with external input), NAR networks (neural auto-regressive), and time delay neural networks. It follows a similar format to the neural fitting (nftool), clustering (nctool), and pattern recognition (nprtool) tools.
Network diagrams shown in the Neural Time Series Tool, Neural Training Tool, and with the view(net) command, have been improved to show tap delay lines in front of weights, the sizes of inputs, layers and outputs, and the time relationship of inputs and outputs. Open loop feedback outputs and inputs are indicated with matching tab and indents in their respective blocks.
The Save Results panel of the Neural Network Time Series Tool allows you to generate both a Simple Script, which demonstrates how to get the same results as were obtained with the wizard, and an Advanced Script, which provides an introduction to more advanced techniques.
The Train Network panel of the Neural Network Time Series Tool introduces four new plots, which you can also access from the Network Training Tool and the command line. The error histogram of any static or dynamic network can be plotted.

plotresponse(errors)

The dynamic response can be plotted, with colors indicating how targets were assigned to training, validation and test sets across timesteps. (Dividing data by timesteps and other criteria, in addition to by sample, is a new feature described in New Time Series Validation on page 13.)

plotresponse(targets,outputs)
The autocorrelation of error across varying lag times can be plotted.

ploterrcorr(errors)

The input-to-error correlation can also be plotted for varying lags.
plotinerrcorr(inputs,errors)
Simpler time series neural network creation is provided for NARX and time-delay networks, and a new function creates NAR networks. All the network diagrams shown here are generated with the command view(net).
net = narxnet(inputDelays, feedbackDelays, hiddenSizes, feedbackMode, trainingFcn net = narnet(feedbackDelays, hiddenSizes, feedbackMode, trainingFcn) net = timedelaynet(inputDelays, hiddenSizes, trainingFcn)
Several new data sets provide sample problems that can be solved with these networks. These data sets are also available within the ntstool GUI and the command line.
[x, [x, [x, [x, [x, [x, [x, [x, [x, t] t] t] t] t] t] t] t] t] = = = = = = = = = simpleseries_dataset; simplenarx_dataset; exchanger_dataset; maglev_dataset; ph_dataset; pollution_dataset; refmodel_dataset; robotarm_dataset; valve_dataset;
The preparets function formats input and target time series for time series networks, by shifting the inputs and targets as needed to fill initial input and layer delay states. This function simplifies what is normally a tricky data preparation step that must be customized for details of each kind of network and its number of delays.
[x, t] = simplenarx_dataset; net = narxnet(1:2, 1:2, 10); [xs, xi, ai, ts] = preparets(net, x, {}, t); net = train(net, xs, ts, xi, ai); y = net(xs, xi, ai)
The output-to-input feedback of NARX and NAR networks (or custom time series network with output-to-input feedback loops) can be converted between open- and closed-loop modes using the two new functions closeloop and openloop.
net = narxnet(1:2, 1:2, 10);
net = closeloop(net) net = openloop(net)
The total delay through a network can be adjusted with the two new functions removedelay and adddelay. Removing a delay from a NARX network which has a minimum input and feedback delay of 1, so that it now has a minimum delay of 0, allows the network to predict the next target value a timestep ahead of when that value is expected.
net = removedelay(net) net = adddelay(net)
The new function catsamples allows you to combine multiple time series into a single neural network data variable. This is useful for creating input and target data from multiple input and target time series.
x = catsamples(x1, x2, x3); t = catsamples(t1, t2, t3);
In the case where the time series are not the same length, the shorter time series can be padded with NaN values. This will indicate dont care or equivalently dont know input and targets, and will have no effect during simulation and training.
x = catsamples(x1, x2, x3, 'pad') t = catsamples(t1, t2, t3, 'pad')

Alternatively, the shorter series can be padded with any other value, such as zero.
x = catsamples(x1, x2, x3, 'pad', 0)
There are many other new and updated functions for handling neural network data, which make it easier to manipulate neural network time series data.

help nndatafun

New Time Series Validation
Normally during training, a data sets targets are divided up by sample into training, validation and test sets. This allows the validation set to stop training at a point of optimal generalization, and the test set to provide an independent measure of the networks accuracy. This mode of dividing up data is now indicated with a new property:
net.divideMode = 'sample'
However, many time series problems involve only a single time series. In order to support validation you can set the new property to divide data up by timestep. This is the default setting for NARXNET and other time series networks.

net.divideMode = 'time'

This property can be set manually, and can be used to specify dividing up of targets across both sample and timestep, by all target values (i.e., across sample, timestep, and output element), or not to perform data division at all.
net.divideMode = 'sampletime' net.divideMode = 'all' net.divideMode = 'none'
New Time Series Properties
Time series feedback can also be controlled manually with new network properties that represent output-to-input feedback in open- or closed-loop
modes. For open-loop feedback from an output from layer i back to input j, set these properties as follows:
net.inputs{j}.feedbackOutput = i net.outputs{i}.feedbackInput = j net.outputs{i}.feedbackMode = 'open'
When the feedback mode of the output is set to 'closed', the properties change to reflect that the output-to-input feedback is now implemented with internal feedback by removing input j from the network, and having output properties as follows:
net.outputs{i}.feedbackInput = []; net.outputs{i}.feedbackMode = 'closed'
Another output property keeps track of the proper closed-loop delay, when a network is in open-loop mode. Normally this property has this setting:
net.outputs{i}.feedbackDelay = 0
However, if a delay is removed from the network, it is updated to 1, to indicate that the networks output is actually one timestep ahead of its inputs, and must be delayed by 1 if it is to be converted to closed-loop form.
net.outputs{i}.feedbackDelay = 1
New Flexible Error Weighting and Performance
Performance functions have a new argument list that supports error weights for indicating which target values are more important than others. The train function also supports error weights.

net = train(net, x, t, xi, ai, ew) perf = mse(net, x, t, ew)
You can define error weights by sample, output element, time step, or network output:
ew ew ew ew = = = = [1.0 0.5 0.7 0.2]; [0.1; 0.5; 1.0]; {0.1 0.2 0.3 0.5 1.0}; {1.0; 0.5}; % % % % Weighting errors across 4 samples. across 3 output elements. across 5 timesteps. across 2 network outputs
These can also be defined across any combination. For example, weighting error across two time series (i.e., two samples) over four timesteps:
ew = {[0.5 0.4], [0.3 0.5], [1.0 1.0], [0.7 0.5]};
In the general case, error weights can have exactly the same dimension as targets, where each target has an associated error weight. Some performance functions are now obsolete, as their functionality has been implemented as options within the four remaining performance functions: mse, mae, sse, and sae. The regularization implemented in msereg and msnereg is now implemented with a performance property supported by all four remaining performance functions.
% Any value between the default 0 and 1. net.performParam.regularization
The error normalization implemented in msne and msnereg is now implemented with a normalization property.
% Either 'normalized', 'percent', or the default 'none'. net.performParam.normalization
A third performance parameter indicates whether error weighting is applied to square errors (the default for mse and sse) or the absolute errors (mae and sae).
net.performParam.squaredWeighting % true or false
Compatibility Considerations
The old performance functions and old performance arguments lists continue to work as before, but are no longer recommended.
New Real Time Workshop and Improved Simulink Support
Neural network Simulink blocks now compile with Real Time Workshop and are compatible with Rapid Accelerator mode.
gensim has new options for generating neural network systems in Simulink. Name - the system name SampleTime - the sample time InputMode - either port, workspace, constant, or none. OutputMode - either display, port, workspace, scope, or none SolverMode - either default or discrete

% New function net = feedforwardnet(hiddenSizes, trainingFcn) % Old function net = newff(x,t,hiddenSizes, transferFcns, trainingFcn,. learningFcn, performanceFcn, inputProcessingFcns,. outputProcessingFcns, dataDivisionFcn)
The new functions (and the old functions they replace) are:
feedforwardnet (newff) cascadeforwardnet (newcf) competlayer (newc) distdelaynet (newdtdnn) elmannet (newelm) fitnet (newfit) layrecnet (newlrn) linearlayer (newlin) lvqnet (newlvq) narxnet (newnarx, newnarxsp) patternnet (newpr) perceptron (newp) selforgmap (newsom) timedelaynet (newtdnn)
The networks inputs and outputs are created with size zero, then configured for data when train is called or by optionally calling the new function configure.
net = configure(net, x, t)
Unconfigured networks can be saved and reused by configuring them for many different problems. unconfigure sets a configured networks inputs and outputs to zero, in a network which can later be configured for other data.

net = unconfigure(net)

Old functions continue working as before, but are no longer recommended.

Improved GUIs

The neural fitting nftool, pattern recognition nprtool, and clustering nctool GUIs have been updated with links back to the nnstart GUI. They give the option of generating either simple or advanced scripts in their last panel. They also confirm with you when closing, if a script has not been generated, or the results not yet saved.
Improved Memory Efficiency
Memory reduction, the technique of splitting calculations up in time to reduce memory requirements, has been implemented across all training algorithms for both gradient and network simulation calculations. Previously it was only supported for gradient calculations with trainlm and trainbr. To set the memory reduction level, use this new property. The default is 1, for no memory reduction. Setting it to 2 or higher splits the calculations into that many parts.
net.efficiency.memoryReduction
The trainlm and trainbr training parameter MEM_REDUC is now obsolete. References to it will need to be updated. Code referring to it will generate a warning.

Improved Data Sets

All data sets in the toolbox now have help, including example solutions, and can be accessed as functions:
help simplefit_dataset [x, t] = simplefit_dataset;
See help for a full list of sample data sets:

help nndatasets

Updated Argument Lists
The argument lists for the following types of functions, which are not generally called directly, have been updated. The argument list for training functions, such as trainlm, traingd, etc., have been updated to match train. The argument list for the adapt function adaptwb has been updated. The argument list for the layer and network initialization functions, initlay, initnw, and initwb have been updated.

Any custom functions of these types, or code which calls these functions manually, will need to be updated.
Version 6.0.4 (R2010a) Neural Network Toolbox Software
This table summarizes whats new in Version 6.0.4 (R2010a). New Features and Changes No Version Compatibility Considerations No Fixed Bugs and Known Problems Bug Reports Includes fixes
Version 6.0.3 (R2009b) Neural Network Toolbox Software
This table summarizes whats new in Version 6.0.3 (R2009b). New Features and Changes No Version Compatibility Considerations No Fixed Bugs and Known Problems Bug Reports Includes fixes
Version 6.0.2 (R2009a) Neural Network Toolbox Software
This table summarizes whats new in Version 6.0.2 (R2009a). New Features and Changes No Version Compatibility Considerations No Fixed Bugs and Known Problems Bug Reports Includes fixes
Version 6.0.1 (R2008b) Neural Network Toolbox Software
This table summarizes whats new in Version 6.0.1 (R2008b). New Features and Changes No Version Compatibility Considerations No Fixed Bugs and Known Problems Bug Reports Includes fixes
Version 6.0 (R2008a) Neural Network Toolbox Software
This table summarizes whats new in Version 6.0 (R2008a): New Features and Changes Yes Details below Version Compatibility Considerations YesDetails labeled as Compatibility Considerations, below. See also Summary. Fixed Bugs and Known Problems Bug Reports Includes fixes
New features and changes introduced in this version are: New Training GUI with Animated Plotting Functions on page 26 New Pattern Recognition Network, Plotting, and Analysis GUI on page 27 New Clustering Training, Initialization, and Plotting GUI on page 27 New Network Diagram Viewer and Improved Diagram Look on page 28 New Fitting Network, Plots and Updated Fitting GUI on page 28
New Training GUI with Animated Plotting Functions
Training networks with the train function now automatically opens a window that shows the network diagram, training algorithm names, and training status information. The window also includes buttons for plots associated with the network being trained. These buttons launch the plots during or after training. If the plots are open during training, they update every epoch, resulting in animations that make understanding network performance much easier. The training window can be opened and closed at the command line as follows:
nntraintool nntraintool('close')
Two plotting functions associated with the most networks are: plotperformPlot performance. plottrainstatePlot training state.
To turn off the new training window and display command-line output (which was the default display in previous versions), use these two training parameters:

net.trainParam.showWindow = false; net.trainParam.showCommandLine = true;
New Pattern Recognition Network, Plotting, and Analysis GUI
The nprtool function opens a GUI wizard that guides you to a neural network solution for pattern recognition problems. Users can define their own problems or use one of the new data sets provided. The newpr function creates a pattern recognition network at the command line. Pattern recognition networks are feed-forward networks that solve problems with Boolean or 1-of-N targets and have confusion (plotconfusion) and receiver operating characteristic (plotroc) plots associated with them. The new confusion function calculates the true/false, positive/negative results from comparing network output classification with target classes.
New Clustering Training, Initialization, and Plotting GUI
The nctool function opens a GUI wizard that guides you to a self-organizing map solution for clustering problems. Users can define their own problem or use one of the new data sets provided. The initsompc function initializes the weights of self-organizing map layers to accelerate training. The learnsomb function implements batch training of SOMs that is orders of magnitude faster than incremental training. The newsom function now creates a SOM network using these faster algorithms.
Several new plotting functions are associated with self-organizing maps: plotsomhitsPlot self-organizing map input hits. plotsomncPlot self-organizing map neighbor connections. plotsomndPlot self-organizing map neighbor distances. plotsomplanesPlot self-organizing map input weight planes. plotsomposPlot self-organizing map weight positions. plotsomtopPlot self-organizing map topology.
You can call the newsom function using conventions from earlier versions of the toolbox, but using its new calling conventions gives you faster results.
New Network Diagram Viewer and Improved Diagram Look
The new neural network diagrams support arbitrarily connected network architectures and have an improved layout. Their visual clarity has been improved with color and shading. Network diagrams appear in all the Neural Network Toolbox graphical interfaces. In addition, you can open a network diagram viewer of any network from the command line by typing

view(net)

New Fitting Network, Plots and Updated Fitting GUI
The newfit function creates a fitting network that consistes of a feed-forward backpropagation network with the fitting plot (plotfit) associated with it. The nftool wizard has been updated to use newfit, for simpler operation, to include the new network diagrams, and to include sample data sets. It now allows a Simulink block version of the trained network to be generated from the final results panel.
The code generated by nftool is different the code generated in previous versions. However, the code generated by earlier versions still operates correctly.

Automated Data Division During Network Creation
When training with supervised training functions, such as the Levenberg-Marquardt backpropagation (the default for feed-forward networks), you can supply three sets of input and target data. The first data set trains the network, the second data set stops training when generalization begins to suffer, and the third data set provides an independent measure of network performance. Automated data division occurs during network creation in the Network/Data Manager GUI, Neural Network Fitting Tool GUI, and at the command line. At the command line, to create and train a network with early stopping that uses 20% of samples for validation and 20% for testing, you can use the following code:
net = newff(p,t,20); net = train(net,p,t);
Previously, you entered the following code to accomplish the same result:
pr = minmax(p); s2 = size(t,1); net = newff(pr,[20 s2]); [trainV,validateV,testV] = dividevec(p,t,0.2,0.2); [net,tr] = train(net,trainV.P,trainV.T,[],[],validateV,testV);
For more information about data division, see Multilayer Networks and Backpropagation Training in the Neural Network Toolbox Users Guide.
New Data Division Functions
The following are new data division functions: dividerandDivide vectors using random indices. divideblockDivide vectors in three blocks of indices. divideintDivide vectors with interleaved indices. divideindDivide vectors according to supplied indices.
Default Data Division Settings
Network creation functions return the following default data division properties: net.divideFcn = 'dividerand' net.divedeParam.trainRatio = 0.6; net.divideParam.valRatio = 0.2; net.divideParam.testRatio = 0.2; Calling train on the network object net divided the set of input and target vectors into three sets, such that 60% of the vectors are used for training, 20% for validation, and 20% for independent testing.
Changing Default Data Division Settings
You can override default data division settings by either supplying the optional data division argument for a network-creation function, or by changing the corresponding property values after creating the network. After creating a network, you can view and modify the data division behavior using the following new network properties: net.divideFcnName of the division function net.divideParamParameters for the division function
New Simulink Blocks for Data Preprocessing
New blocks for data processing and reverse processing are available. For more information, see Processing Blocks in the Neural Network Toolbox Users Guide. The function gensim now generates neural networks in Simulink that use the new processing blocks.
Properties for Targets Now Defined by Properties for Outputs

The properties for targets are now defined by the properties for outputs. Use the following properties to get and set the output and target properties of your network: net.numOutputsThe number of outputs and targets net.outputConnectIndicates which layers have outputs and targets net.outputsCell array of output subobjects defining each output and its target
Several properties are now obsolete, as described in the following table. Use the new properties instead. Recommended Property
net.numOutputs net.outputConnect net.outputs

Obsolete Property

net.numTargets net.targetConnect net.targets
Version 5.0.2 (R2007a) Neural Network Toolbox Software
This table summarizes whats new in Version 5.0.2 (R2007a): New Features and Changes No Version Compatibility Considerations No Fixed Bugs and Known Problems Bug Reports
Version 5.0.1 (R2006b) Neural Network Toolbox Software
This table summarizes whats new in Version 5.0.1 (R2006b): New Features and Changes No Version Compatibility Considerations No Fixed Bugs and Known Problems Bug Reports
Version 5.0 (R2006a) Neural Network Toolbox Software
This table summarizes whats new in Version 5.0 (R2006a): New Features and Changes Yes Details below Version Compatibility Considerations YesDetails labeled as Compatibility Considerations, below. See also Compatibility Considerations on page 42. Fixed Bugs and Known Problems Bug Reports
New features and changes introduced in this version are Dynamic Neural Networks on page 40 Wizard for Fitting Data on page 41 Data Preprocessing and Postprocessing on page 41 Derivative Functions Are Obsolete on page 42

Dynamic Neural Networks

Version 5.0 now supports these types of dynamic neural networks:
Time-Delay Neural Network
Both focused and distributed time-delay neural networks are now supported. Continue to use the newfftd function to create focused time-delay neural networks. To create distributed time-delay neural networks, use the newdtdnn function.
Nonlinear Autoregressive Network (NARX)
To create parallel NARX configurations, use the newnarx function. To create series-parallel NARX networks, use the newnarxsp function. The sp2narx function lets you convert NARX networks from series-parallel to parallel configuration, which is useful for training.

Layer Recurrent Network (LRN)
Use the newlrn function to create LRN networks. LRN networks are useful for solving some of the more difficult problems in filtering and modeling applications.

Custom Networks

The training functions in Neural Network Toolbox are enhanced to let you train arbitrary custom dynamic networks that model complex dynamic systems. For more information about working with these networks, see the Neural Network Toolbox documentation.

Wizard for Fitting Data

The new Neural Network Fitting Tool (nftool) is now available to fit your data using a neural network. The Neural Network Fitting Tool is designed as a wizard and walks you through the data-fitting process step by step. To open the Neural Network Fitting Tool, type the following at the MATLAB prompt:

nftool

Data Preprocessing and Postprocessing
Version 5.0 provides the following new data preprocessing and postprocessing functionality:
dividevec Automatically Splits Data
The dividevec function facilitates dividing your data into three distinct sets to be used for training, cross validation, and testing, respectively. Previously, you had to split the data manually.
fixunknowns Encodes Missing Data
The fixunknowns function encodes missing values in your data so that they can be processed in a meaningful and consistent way during network training. To reverse this preprocessing operation and return the data to its original state, call fixunknowns again with 'reverse' as the first argument.
removeconstantrows Handles Constant Values
removeconstantrows is a new helper function that processes matrices by removing rows with constant values.
mapminmax, mapstd, and processpca Are New
The mapminmax, mapstd, and processpca functions are new and perform data preprocessing and postprocessing operations. Compatibility Considerations. Several functions are now obsolete, as described in the following table. Use the new functions instead. New Function

mapminmax

Obsolete Functions
premnmx postmnmx tramnmx prestd poststd trastd prepca trapca

mapstd

processpca

Each new function is more efficient than its obsolete predecessors because it accomplishes both preprocessing and postprocessing of the data. For example, previously you used premnmx to process a matrix, and then postmnmx to return the data to its original state. In this release, you accomplish both operations using mapminmax; to return the data to its original state, you call mapminmax again with 'reverse' as the first argument:
mapminmax('reverse',Y,PS)
Derivative Functions Are Obsolete
The following derivative functions are now obsolete:
ddotprod dhardlim dhardlms dlogsig
dmae dmse dmsereg dnetprod dnetsum dposlin dpurelin dradbas dsatlin dsatlins dsse dtansig dtribas
Each derivative function is named by prefixing a d to the corresponding function name. For example, sse calculates the network performance function and dsse calculated the derivative of the network performance function.
To calculate a derivative in this version, you must pass a derivative argument to the function. For example, to calculate the derivative of a hyperbolic tangent sigmoid transfer function A with respect to N, use this syntax:
A = tansig(N,FP) dA_dN = tansig('dn',N,A,FP)
Here, the argument 'dn' requests the derivative to be calculated.
Version 4.0.6 (R14SP3) Neural Network Toolbox Software
This table summarizes whats new in Version 4.0.6 (R14SP3): New Features and Changes No Version Compatibility Considerations No Fixed Bugs and Known Problems Bug Reports
Compatibility Summary for Neural Network Toolbox Software
This table summarizes new features and changes that might cause incompatibilities when you upgrade from an earlier version, or when you use files on multiple versions. Details are provided with the description of the new feature or change. Version (Release) Latest Version V7.0.1 (R2011a) V7.0 (R2010b) New Features and Changes with Version Compatibility Impact None See the Compatibility Considerations subheading for this new feature or change: New Flexible Error Weighting and Performance on page 14 Improved Network Creation on page 18 Improved Memory Efficiency on page 20 Updated Argument Lists on page 21 V6.0.4 (R2010a) V6.0.3 (R2009b) V6.0.2 (R2009a) V6.0.1 (R2008b) None None None None
Version (Release) V6.0 (R2008a)
New Features and Changes with Version Compatibility Impact See the Compatibility Considerations subheading for this new feature or change: New Training GUI with Animated Plotting Functions on page 26 New Clustering Training, Initialization, and Plotting GUI on page 27 New Fitting Network, Plots and Updated Fitting GUI on page 28