We know excessive the problem that plans to close, this is you actually the model that compose builds duplicated to train data result, suit completely to train data, did not generalize to represent data source better, provide new data when you have disastrous result and achieve very odd result.
When compose builds machine study model, the data that makes spent likelihood much from beginning to end. I think we realized excessive the problem that plans to close, it is the result that the model that you build duplicated to train data substantially, so perfect ground and be identical of training data phase, is not extensive is changed those who will represent data better is overall, when input new data when you and getting very odd result, can produce disastrous result.
What can you do to prevent this kind of circumstance happening?
Online examine, your meeting discovery is concerned excessive plan to close, it is what and the many literary works that how prevent it. These had been finished greatly, so I won't repeat these here, but before beginning normally, pass separate available data, can control them easily.
What data needs to keep apart?
The basiccest method is the example that get groups of whole available data and differs its fractionation for 3 (did not repeat) give out the following label:
Each groups of training, test and data set of test and verify have specific objective, and won't have identical volume normally. Actually, a few the most commonly used values are 3: 1: 1 break up (or test and verify of 60 ％ training, 20 ％ and 20 ％ test)
Train data
This is the core of data. This is the content that is used at training and the model that establish you to use. Of course, the data that you obtain in training is more, you can expect the model undertakes to its better wraparound. This is occupied normally among them the about 60 ％ of all available data.
Data of test and verify
This is the data that is used at tone actors or actress and improving a model. Whenever train a model, want to use its at collect of test and verify, so that generate a few forecast and undertake grading to its function. Next, you can adjust a model, train afresh and move in the light of this data again, whether to already undertake in order to examine improvement. This may be the measure of a special iteration. You need a reasonable data size to cover the parameter space that the model will expose, but not too much, so that your model does not have enough data,will train. This occupies the 20 ％ of all available data about normally.
Test data
These data are laid aside in the begining in the project aside, till you the ability when the model to you is satisfactory can be handled. Next, you apply its hereat data, whether to learn in order to examine it to be generalized very well really, or it whether by excessive plan to close. As similar as data set of test and verify, you need enough data to enclothe you to want the content of the test, ensure your model can work normally, but you do not hope to get too much information from inside the content that can train. This is the 20 ％ left and right sides of available data normally.
Were you mentioned cover parameter space?
I was accomplished. For example, if you want to be aimed at the discharge of different conduit diameter to run a model, you need to ensure the know exactly about sth in test and verify is occupied (it is these 3 groups of all data actually) in order to ensure the model can learn this function and have a test to it. If you train the data of diameter of a conduit only, so the chance that the relation exists between its study current and tubal diameter is very little, the effect that causes other canal diameter thereby is very poor.
Accordingly, it is very important that the data that keeps enough is used at training.
Save data
If the data of 40 ％ is used at tone actors or actress and undertaking final grading to the model only, so complex to model, you may need much data, and this model needs the character with a large number of very big change again. To add the data bulk that can be used at training data, the scientist can use alternate test and verify normally.
Alternate test and verify is to will train medium data and assemble of fixed collect of test and verify to rise, divide into systematically next paragraph (or " fold " ) , assign every team next, is not secure them center in training and test and verify. For example, to test and verify of a fourfold across, the data that is used at test and test and verify is divided into 4 equal groups, every group regards test and verify as collect by turns, other 3 assembles are incorporated training concentration.
Final, this means you to had counted foothold to undertake forecasting to every, use its at training a model (but not be to undertake at the same time) , and using more data. Of the specific model that alternate test and verify still can adopt shift of this kind of fashion to actor or actress you are being used exceed parameter, in order to raise precision.
So what is the problem?
You still use the data of 80 ％ only, and always be the result that market of hope use test will come to examine current model. This is not a problem normally, but premise is you master the data that how uses place to obtain.
Some people will train and adjust a model, compare collect of test and verify and the accuracy that test market, when the accuracy that testing market rises to become poorer however no longer, stop to adjust a model. Fall in these circumstances normally, although check collect to become more flooey, model becomes excessive plan to close, device of test and verify also will continue to improve.
The rate that the problem is using test market to check a model at you is too slow, of course, you also made the case of spirits of afore-mentioned way of part of test and verify to testing market really. To poorer data or very complex model, you may run it in the light of test market for many times, and won't realize you are excessive plan to close, the information that because you are using you,sees undertakes changing to the model.
For example, you get final model and it moves on test market, although the result is no problem, but if change a parameter,you can notice it may get ameliorative. You change it, the model got improvement, but the information that you already used concerned test part will provide a model. Each already divulged information between the group.
Information leak
Information leak is to show the information leak that comes from sealed system arrives it not should the place of leak. This is a terminology that via commonly used Yu Mi the code learns, among them safe system provides the clue about in-house personnel to listener-in, this may be used at destroying security finally.
So the conduit here is data scientist, he uses the knowledge of themselves and mastery of a skill or technique inadvertently, see according to them its expression to testing market adjusts a model slowly.
I should be a bit more compensatory, this is not a big question normally, I had seen such case 9 times. My method also is not the only method that controls it, but I gave out below my opinion.
What can you do?
Once you realize this may be a problem, this is very easy pilot. The simplest solution is to ensure you did not use ultimate test part for many times, change according to restricting what what you do as a result.
Move actor to the model, actual take out another small rally from inside training and pool of data of test and verify, use its as the representative that tests market may be beneficial. Carry this kind of kind, you still can optimize your model, the improvement that examines a model is how undertake (add here test market becomes more stop when flooey) , and reserve ultimate test market written guarantee finally in order to show the advantage of fruit.
Although this sounds,took away more data from inside training, but method of alternate test and verify can be helped alleviate this kind of circumstance. For example, you can use the 10 ％ in 80 ％ to arrive 20 ％ , and you still are opposite 60 ％ reach the rest 70 ％ undertake training and test and verify (this and method of blame across test and verify are similar) .
Bit more crucial
Paul May is adviser of science of a data, devote oneself to to amass extraction value from two kinds of available data, combine other and novel origin to enhance its effectiveness at the same time. He uses machine study to technology and statistical analysis method will handle and provide data to all sorts of origin.
