Expert: Saurabh Kudesia - 8/10/2004

Question
I'm a bit confused by your response that you have already answered my followup question. In response to my initial question, you requested clarification:
(http://www.allexperts.com/answerv.asp?QuestionID=3640687).

I'm afraid I didn't receive any response to my more specific followup: (http://www.allexperts.com/answerv.asp?QuestionID=3651844)

If the followup was still too vague to answer, I apologize; perhaps I can ask another time when I understand the subject better.

In any case, thank you for taking the time to respond.


-------------------------
Followup To
Question -
Thank you for the response.

(In mathematics, think I can handle basic statistics, algebra and summations, and maybe integrals and derivitives. As long as you can provide some online references, I'll try to puzzle out any formulas or theories you would like to describe.)

When you have input and output from a function, but are ignorant of how the function works, what mathematical methods can help you determine:
A) Is a neural net is appropriate for the problem?
B) Do you have the computing power to train and run such a NN?
C) Which type of NN, training method, and parameters should you use?

If possible, I would like to use the following hypothetical situation to understand how one calculates these things. (I suspect this is a good example of a problem for which one should /not/ use a NN, so I don't intend to actually create such a NN.)
---------------
N,M, B, and X are integer constants

An "uncompressed frame" represents an array of N B-bit signed integers with a high level of redundancy in both adjacent elements and in their frequencies. (e.g. a fragment of sound from a song)

A "compressed frame" represents a list of M 8-bit values with a very low level of redundancy in both adjacent elements and in their frequencies.

Knowledge of the compression algorithm is not available. Only simple mathematical operations can be used to move data in and out of the NN. (e.g. divide a 16-bit integer by 2^15 to obtain a node value of -1 to 1)

During training time, an "example" consists of an uncompressed frame and the corresponding compressed frame. The training set contains up to X relevant examples. (e.g. fragments from various songs, but not unmusical noise)

During run time, a compressed frame is presented to the NN. It is expected to compute a frame that is very close to the corresponding uncompressed frame.

The error rate for uncompressing the training set must be less than E, where an "error" is defined as an element of the computed frame differing by more than one from the corresponding element in the original uncompressed frame.

Some possible values:
N=8192
M=2048
B=16
X=1048576
E=1/131072
---------------

Answer -  

Answer
Dear JP,

There were lot of things that you coupled together with your question that makes the entire question vague. Let me tell you why?

When you have input and output from a function, but are ignorant of how the function works, Don't assume anything. This is because the entire process is totally unknown, many different processes can give you the same output with the same input. But what about the underlying logic that forms the process. When you don't have any logic- you just can not train NN to form such logic, because if you do so, every process will be correct if it joins the inputs and outputs. That is certainly not acceptable. The NN may go into infinite loop trying out all the possible combination and still may not have the sufficient capacity to fix the problem. Secondly, this would again required unknown computing power since we are not aware how much steps or time does this process will take. That's make the whole NN baseless and confusing.

Yes, we do have sufficient computing power to train and run such NNs. But how much computing power will this NN requires? That again depends on the complexity of the problem. Computing power needed to solve simple algebraic question will much lesser than that required for modelling the entire human brain.

Training methods and parameters varies with problem. It depends on the number of variable you are using in a equation, the inter relationship between variable and the related steps. Let me give you an simple example:

X=2y
y=2X

Can you find out the value of X, y? Both of these equation could be an intermediate steps in any calculations or it could be just the criteria to judge the correctness of the answer at two different stages of problem solving. In both of the above equation, the other related parameters are not given, making it slightly difficult to solve. But if I tell you that these two equations are just two steps in a process, you have better change to solve the problem. NN training for such equations will therefore be completley different than compared to other algebraic solutions. There fore, training parameters and solutions differs widely.

I hope that now you can understand how to isolate one set of problem with other and find a solution for it. If you try to reply to your question as just one, it is definitly vague. I do'nt want to confuse you on this issue and therefore I choose this option to respond.

If you have any other questions or queries, Pl. feel free to post it. I would be honoured to help you in every possible way.

All the very best
Regards
Saurabh