WGPower = @(x) ((sin(m*pi*x/a)).^2.*(cos(n*pi*y/b)).^2)+((cos(m*pi*x/a)).^2.*(sin(n*pi*y/b)).^2);
However, this is going to use the linspace value of y from the scope it is defined in.
Then when you have
[pos,fmin]=fminsearch(WGPower,[0.01 0.01]);
the [0.01 0.01] tells fminsearch that it is searching over a vector of length 2. So all of the vectors that it passes in to "x" in the anonymous function will be length 2. And that is not going to work with the length 10 value of y.
My thought is that probably what you are after is
WGPower = @(xy) ((sin(m*pi*xy(1)/a)).^2.*(cos(n*pi*xy(2)/b)).^2)+((cos(m*pi*xy(1)/a)).^2.*(sin(n*pi*xy(2)/b)).^2);
Take note that the "x" and "y" linspace done just before the assignment are not going to have any effect on anything.
You should also be looking more closely at your fprintf() statement about the position of the minimum. As your x0 is length 2, the position of the minimum is going to be length 2. That fprintf() format will accept that, and it will deal with it by printing out "The position" text twice, once for pos(1) and once for pos(2) . Perhaps you want to use '%2.4f %2.4f' instead of just one %2.4f
