We have observed the millisecond X-ray pulsar SAX J1808.4-3658 on three occasions during its 2000 outburst with the BeppoSAX satellite. The source was highly variable and erratic during this outburst, and by coincidence we obtained data only during times when the source had very low luminosities. During our observations, we detected four faint sources. The source closest to the position of SAX J1808.4-3658 is still ~1.6 away. This source can only be identified with SAX J1808.4-3658 if we assume that the BeppoSAX positional reconstruction is not completely understood. We also reanalyzed a BeppoSAX observation taken in March 1999 when the source was in quiescence and during which the source was thought to have been detected (Stella et al. 2000). Based on the similarities (position and luminosity) of this source with the above mentioned source ~1.6 away from SAX J1808.4-3658, it is possible that they are the same source. If this source is not the millisecond pulsar, then during all BeppoSAX observations of SAX J1808.4-3658 (the 2000 outburst ones and the 1999 quiescent one), the millisecond pulsar was not detected. A reanalysis of the ASCA quiescent data of SAX J1808.4-3658 (Dotani, Asai, & Wijnands 2000) confirms that during this observation the source was securely detected in quiescence. We discuss our results for SAX J1808.4-3658 in the context of the quiescent properties of low-mass X-ray binary transients.