The study of stochastic queueing networks is quite important due to the many applications including transportation, telecommunication, and manufacturing industries. Since there is often no explicit solution to these types of control problems, numerical methods are needed. Following the method of Boué-Dupuis, we use a Dynamic Programming approach of optimization on a controlled Markov Chain that simulates the behavior of a fluid limit of the original process. The search for an optimal control in this case involves a Skorokhod problem to describe the dynamics on the boundary of closed, convex domain. Using relaxed stochastic controls we show that the approximating numerical solution converges to the actual solution as the size of the mesh in the discretized state space goes to zero, and illustrate with an example.