In this paper, we propose an operational space control formulation for a planar N-link underactuated manipulator ($PA^{N-1}$) with a passive first joint subject to actuator constraints (N is greater than 3 or equal to 3), covering both stabilization and tracking tasks. Such underactuated manipulators have an inherent first-order nonholonomic constraint, allowing us to project their dynamics to a space consistent with the nonholonomic constraint. Based on the constrained dynamics, we can design operational space controllers with respect to tasks assuming that all joints of the manipulator are active. Due to underactuation, we design a Quadratic Programming (QP) based controller to minimize the error between the desired torque commands and available motor torques in the null space of the constraint, as well as involve the constraint of motor outputs. The proposed control framework was demonstrated by stabilization and tracking tasks in simulations with both planar PA2 and PA3 manipulators. Furthermore, we verified the controller experimentally using a planar PA2 robot.