What is the error in following program in GW BASIC?

Follow the logic:

1. You enter a number, a.
2. The program creates b as 1.
3. The program immediately adds 1 to b, so that b is now 2.
4. The program sets k to a/b. This means that k is now half of a.
5. The program sets p to k without the .5 that may or may not be there.
6. If p (half of a rounded down) is equal to k (half of a not rounded down), that is, if a is divisible by b, it goes to 100 and says it is not a prime number.
7. Otherwise, if b (which is 2) is less than a the program goes to line 30 and adds another 1 to b and repeats the process, but now b is 3.
8. Otherwise, if b (which is 2) is equal to a or greater than it, it prints that this is a prime number.

Let's say that the number you enter is 2. Two is, in fact, a prime number. Follow the logic above, however, and you will see that the program will tell us that it is not a prime number, because at step 6, two divided by two is one, and one truncated is still one. The same should be true for any number except 1, because all numbers are divisible by themselves.

My guess is that in your testing, you never tested 1; the program should say that 1 is a prime number (this is because even though 1 is divisible by itself, you start at b=2).

(Note also that this is also technically wrong: one is not a prime number.)

This code describes determining a prime number in basic:

10 INPUT p
20 FOR l = 2 TO INT(SQR(p))
30 LET a = p / l
40 LET b = FIX(a)
50 IF a = b THEN GOTO 80
60 NEXT l
70 PRINT "prime"
80 END

The keyword LET is optional in GW-BASIC (even in the original Dartmouth BASIC).

GW-BASIC has a modulo operator (MOD), so testing if A is divisible by B can be expressed much more concise with IF A MOD B=0 THEN ….

The loop that is expressed with increasing B, testing B and jumping to the loop start with GOTO would be easier to read if it is expressed as an actual FOR loop.

So the original program can be expressed like this:

10 INPUT "What is the number";A
20 FOR B=2 TO A
30  IF A MOD B=0 THEN PRINT "It is not a prime number.":END
40 NEXT
50 PRINT "It is a prime number."

This checks if A is divisible by itself in the last iteration which is always true. So we need to limit the upper limit of the iterations to A-1.

The other problem are numbers <2 for which the loop isn't run at all and are therefore identified as prime numbers. A simple hack to fix this would be testing for this and setting A to a known non-prime number >2.

10 INPUT "What is the number";A
20 IF A<2 THEN A=4:REM So that numbers <2 are reported as non-prime.
30 FOR B=2 TO A-1
40  IF A MOD B=0 THEN PRINT "It is not a prime number.":END
50 NEXT
60 PRINT "It is a prime number."