If we want to show that a program will function in the way we expect it to, one way is to use formal reasoning techniques, known as verification. This means we have mathematical statement we know are true about the properties of the programming language we are using, and we put them together to show that the whole program must behave according to its specification by a full logical argument. This is an important area of research in Computer Science, and it happens that the department at Queen Mary is a world leader in it, you can find more details of this research here. However, scaling up these techniques to large-scale programs is problematical, so most programs go through stages of testing and debugging.
Testing and debugging implies we know there is a possibility of human error in a program, so we run it under various circumstances to see if we can find cases where it does not perform in the way we expect. Debugging is the process of altering the program to make it perform correctly. If our programs have been constructed according to good software engineering practice, they will be divided into parts, each of which has a clear specification so that it is first tested in isolation to make sure it works according to its specification. If we know the component parts of a program work correctly, but the program itself doesn't, then we know the error which requires debugging is in the code which brings the components together. If we know the code which brings the components together works correctly so long as the components work correctly, then we can isolate the problem to a particular component which requires debugging. A well-designed program should not have any unexpected interaction between components, so it should be easy to track down the cause of incorrect performance to a particular component.
While test runs of programs and program components cannot prove the absence of error in the way that formal verification can, careful choice of test data and methodical application of testing will play an important part in giving confidence the program will perform correctly. Testing and debugging should be considered an essential part of software development, and not done haphazardly as an afterthought. In fact it is becoming recognised that a suite of tests should be developed before the code they are testing, and the tests should be done (using some testing tool) throughout the process of software development. This is referred to as Test Driven Development. To some extent, it means the tests become the specification for the code.
An additional aspect of testing and debugging occurs when we have large systems which interact with real world environments (typically human users). While we can write a precise specification of what is required from program components, human users may not know exactly what they want until they see something working. Experiencing an experimental version of a system working may prompt human users to propose different ways they'd rather see it work. Poorly designed interfaces may mislead human users of computer systems so that although the system is working correctky according to its specification, mistakes are made because the human user has an incorrect understanding of the system and cannot easily see how to do what he or she wants with it. This is a vast and growing topic in Computer Science, which we only touch on here to demonstrate that merely getting programs working according to their specification is not all there is to building correct software.
In MIT's 6.170 course, Lecture 5 starts off with some discussion on Java's exception mechanism, but is mainly about testing. Lecture 12 is about debugging.