Kilonovae represent an important electromagnetic counterpart for compact binary mergers, which could become the most commonly detected gravitational wave (GW) source. Follow-up observations, triggered by GW events, of kilonovae are nevertheless difficult due to poor localization by GW detectors and due to their faint near-infrared peak emission that has limited observational capability. We show that the Near-Infrared Camera (NIRCam) on the James Webb Space Telescope (JWST) will be able to detect kilonovae within the relevant GW-detection range of $sim$ 200 Mpc in short ($lesssim$ 12-second) exposure times for a week following the merger. Despite this sensitivity, a kilonova search fully covering a fiducial localized area of $10$ $mbox{deg}^2$ will not be viable with NIRCam due to its limited field of view. However, targeted surveys may be developed to optimize the likelihood of discovering kilonovae efficiently within limited observing time. We estimate that a survey of $10$ $mbox{deg}^2$ focused on galaxies within 200 Mpc would require about 13 hours, dominated by overhead times; a survey further focused on galaxies exhibiting high star-formation rates would require $sim$ 5 hours. The characteristic time may be reduced to as little as $sim$4 hours, without compromising the likelihood of detecting kilonovae, by surveying sky areas associated with 50%, rather than 90%, confidence regions of 3 GW events, rather than a single event. On detection and identification of a kilonova, a limited number of NIRCam follow-up observations could constrain the properties of matter ejected by the binary and the equation of state of dense nuclear matter.