Nick Walker says he would rather earn his Olympia spot than to get a special invite