Fair workload distribution for multi-server systems with pulling strategies

We consider systems with a single queue and multiple parallel servers. Each server fetches a job from the queue immediately after completing its current work. We propose a pulling strategy that aims at achieving a fair distribution of the number of processed jobs among the servers. We show that if the service times are exponentially distributed then our strategy ensures that in the long run the expected difference among the processed jobs at each server is finite while maintaining a reasonable throughput. We give the analytical expressions for the stationary distribution and the relevant stationary performance indices like the throughput and the system's balance. Interestingly, the proposed strategy can be used to control the join-queue length in fork-join queues and the analytical model gives the closed form expression of the performance indices in saturation.