Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/tighten hash usage (fixing Issues #316 and #423) #441

Closed
wants to merge 111 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
111 commits
Select commit Hold shift + click to select a range
9b5d1b7
Attempting to improve the DHT abstraction.
JoshuaGreen Sep 2, 2023
25a9e54
Adding some important documentation.
JoshuaGreen Sep 2, 2023
0f1f1af
Simplifying things -- dhtKey IS a dhtValue, specifically one that als…
JoshuaGreen Sep 3, 2023
ef64367
Might as well eliminate a compiler warning while we're at it.
JoshuaGreen Sep 3, 2023
2b4c821
Making cppcheck happier.
JoshuaGreen Sep 3, 2023
258f055
I guess this is more generic.
JoshuaGreen Sep 3, 2023
1958bb2
Since we want everything to be 0, we don't need the explicit initiali…
JoshuaGreen Sep 3, 2023
3fe00c7
Prefer unsigned integers.
JoshuaGreen Sep 3, 2023
f6d1f70
Fixing some cppcheck style warnings.
JoshuaGreen Sep 3, 2023
a5f410f
Fixing some typos.
JoshuaGreen Sep 3, 2023
de4eeaf
Fixing some more typos.
JoshuaGreen Sep 3, 2023
3fb8b38
Beginning to track dimensions so we can ensure that we don't write ou…
JoshuaGreen Sep 3, 2023
9518402
Matching style.
JoshuaGreen Sep 3, 2023
427e8f7
Lots of simplifications, plus other changes.
JoshuaGreen Sep 3, 2023
d5dcfb6
Restoring the union as there was no strong reason to remove it. We c…
JoshuaGreen Sep 4, 2023
488b13b
Comparing the right upper bound, the one that indicates how much spac…
JoshuaGreen Sep 4, 2023
1967966
The Leng member is an unsigned (short) int, so cast to ensure it matc…
JoshuaGreen Sep 4, 2023
f95c1cd
Let's just use the right format specifier.
JoshuaGreen Sep 4, 2023
7786ef5
Fixing some issues found by VSCode.
JoshuaGreen Sep 4, 2023
591df89
Oops, forgot a term.
JoshuaGreen Sep 4, 2023
87c28c9
Those functions can be static.
JoshuaGreen Sep 4, 2023
1159ebe
Trying to compute the needed size at compile-time.
JoshuaGreen Sep 4, 2023
df21739
Fixing some typos, allowing NULL pointers to be duplicated.
JoshuaGreen Sep 5, 2023
c4ef24d
Holding back on expanding the buffers until I figure out why this is …
JoshuaGreen Sep 5, 2023
69c6c37
Removing
JoshuaGreen Sep 6, 2023
f8ab1c3
Trying to make uses of tabs consistent.
JoshuaGreen Sep 6, 2023
14daa9e
Bumping the required alignment; this allows us to store long doubles …
JoshuaGreen Sep 6, 2023
300ef7f
If the board is filled then we need 64 piece IDs.
JoshuaGreen Sep 6, 2023
19c2284
Going back to assertions.
JoshuaGreen Sep 6, 2023
cedb9af
Some reformatting.
JoshuaGreen Sep 6, 2023
8858f79
Ensuring that fxfAlloc returns properly-aligned pointers.
JoshuaGreen Sep 7, 2023
fbff0db
Might as well parenthesize the argument.
JoshuaGreen Sep 7, 2023
d63e53d
Trying to make freeing match the alignment that we imposed in fxfAllo…
JoshuaGreen Sep 7, 2023
bd82898
Going with the simplest methods for ensuring that returned pointers a…
JoshuaGreen Sep 8, 2023
96e847f
Need to align before checking agains the max size (in general).
JoshuaGreen Sep 8, 2023
c5347f4
Fixing a few more things.
JoshuaGreen Sep 8, 2023
1c43b0a
Catching more errors, and fixing some rather silly masking bugs.
JoshuaGreen Sep 8, 2023
06d16d9
Choosing the appropriate format specifier.
JoshuaGreen Sep 9, 2023
00039b4
An additional sanity check, and no reason to include floating point i…
JoshuaGreen Sep 9, 2023
743f649
Might as well go with the most generic code, even if it's redundant h…
JoshuaGreen Sep 9, 2023
12a354c
Simplifying things a bit, IMO.
JoshuaGreen Sep 12, 2023
cbaf076
Some more assertions.
JoshuaGreen Sep 12, 2023
313cb97
Hmm, those assertions aren't necessary.
JoshuaGreen Sep 12, 2023
3d00a93
Our elements are all unsigned chars, so no reason to complicate things.
JoshuaGreen Sep 13, 2023
5e94a63
Personal preference for ++i.
JoshuaGreen Sep 13, 2023
c5103a6
Might as well properly handle cases where our lengths are > SIZE_MAX.…
JoshuaGreen Sep 14, 2023
724edb0
Explaining some code that is required here but not required elsewhere.
JoshuaGreen Sep 14, 2023
7ec2014
No reason to load the value twice on each loop iteration.
JoshuaGreen Sep 14, 2023
1cb6ae4
Adding another check for SIZE_MAX overflow.
JoshuaGreen Sep 14, 2023
9266dc0
No need to cast until the final assignment.
JoshuaGreen Sep 14, 2023
3a55615
Trying to minimize the memory allocations by allowing for small alloc…
JoshuaGreen Sep 17, 2023
ef55964
Rounding allocations to the alignment required for fxfMINSIZE since w…
JoshuaGreen Sep 18, 2023
6817cf7
Avoiding casting pointers to integers except when (apparently) absolu…
JoshuaGreen Sep 19, 2023
eac38dd
We don't need to restart the whole function.
JoshuaGreen Sep 19, 2023
bcf8434
Making it possible to compile without FXF.
JoshuaGreen Sep 20, 2023
68c696a
Several changes.
JoshuaGreen Sep 20, 2023
7097300
Switching to memcpy. Hopefully this will avoid angering the strict-a…
JoshuaGreen Sep 20, 2023
1dc98ef
Removing some unnecessary casts.
JoshuaGreen Sep 20, 2023
9e70b8b
Trying to avoid assigning a type to the pointers we return.
JoshuaGreen Sep 20, 2023
fec6d93
Let's stop pretending that SetRange and ClrRange are meaningful in th…
JoshuaGreen Sep 20, 2023
2908674
Saving the result of the arithmetic. Even if we're wrong about the p…
JoshuaGreen Sep 20, 2023
1ecde2d
Lots of changes, attempting to avoid violating strict aliasing in any…
JoshuaGreen Sep 21, 2023
28eb69f
We don't need that check.
JoshuaGreen Sep 21, 2023
231273a
We don't need that assertion. If people try to allocate and fail, th…
JoshuaGreen Sep 21, 2023
a1cd0a9
We don't need those #includes since they're in fxf.h.
JoshuaGreen Sep 21, 2023
8a524fe
Small tweaks.
JoshuaGreen Sep 21, 2023
ad2ed90
Never allocate more than the user requested.
JoshuaGreen Sep 22, 2023
ac621ce
We need pointer differences to be accurate, so we can't allocate too …
JoshuaGreen Sep 25, 2023
b10c9c6
Outputting the amount requested as well as the amount we actually tri…
JoshuaGreen Sep 26, 2023
6af5725
Ensuring that these values are correct even if we later change fxfMIN…
JoshuaGreen Sep 28, 2023
33066e4
Ensuring values are sensible.
JoshuaGreen Sep 29, 2023
9a43209
We're requiring the sizes to be nonnegative, so we can simplify the c…
JoshuaGreen Sep 30, 2023
b07e77d
Going with what is apparently the canonical way to get the maximum al…
JoshuaGreen Oct 7, 2023
314acfe
Small tweaks.
JoshuaGreen Oct 7, 2023
213662d
We only need to compute that value once.
JoshuaGreen Oct 7, 2023
99f9193
Adding some asserts.
JoshuaGreen Oct 10, 2023
ecddab6
Adding more asserts.
JoshuaGreen Oct 10, 2023
988659c
Oops, forgot some semicolons.
JoshuaGreen Oct 10, 2023
c251c5d
Expanding on the comment.
JoshuaGreen Oct 13, 2023
3ba6c2f
Making Cppcheck a bit happier.
JoshuaGreen Oct 20, 2023
35256e0
Small tweak, suggested by Cppcheck.
JoshuaGreen Oct 21, 2023
d48c3f0
If we aren't using FXF, anyone who includes this file needs to know w…
JoshuaGreen Oct 22, 2023
caddc53
Adding a sanity check.
JoshuaGreen Oct 26, 2023
0444b32
Adding some more asserts.
JoshuaGreen Nov 7, 2023
829790f
Noting something else that we may want to consider doing.
JoshuaGreen Nov 15, 2023
3d6eac8
Preferring this syntax, as used throughout the rest of the codebase.
JoshuaGreen Nov 17, 2023
3e11bfc
Correcting the comment to use the appropriate example value.
JoshuaGreen Nov 17, 2023
32c0df4
Finally removing the old code.
JoshuaGreen Nov 22, 2023
35fa682
Saving 1, and better explaining where the bound comes from.
JoshuaGreen Nov 23, 2023
9f202a4
Probably improving this bound.
JoshuaGreen Nov 26, 2023
7be78f3
Asserting on one of our assumptions.
JoshuaGreen Nov 26, 2023
facb5e7
Correcting the constant's name.
JoshuaGreen Nov 26, 2023
bece590
Ensuring we have the constants we need.
JoshuaGreen Nov 26, 2023
23005b1
Shifting the sanity check to the .c file.
JoshuaGreen Nov 27, 2023
ec2e09c
Changing this isn't part of the branch.
JoshuaGreen Nov 27, 2023
5c885c3
Adding a TODO.
JoshuaGreen Dec 8, 2023
454665c
Adding some useful comments, making fxfMAXSIZE an enum (to match fxfM…
JoshuaGreen Dec 9, 2023
4cbce9e
Always including the corresponding header, eliminating a warning abou…
JoshuaGreen Dec 9, 2023
8b56652
We don't need a loop since we already know that we're < fxfMAXSIZE (a…
JoshuaGreen Dec 9, 2023
fe4a949
Replacing two checks with a single check against a compile-time const…
JoshuaGreen Dec 10, 2023
39e2827
Using a different hack to eliminate the warning.
JoshuaGreen Dec 10, 2023
bcab92b
Replacing a complicated expression with a macro.
JoshuaGreen Dec 12, 2023
02c90db
Ensuring that these values stay aligned.
JoshuaGreen Dec 13, 2023
7f6c53a
Merging the enums to eliminate a compiler warning.
JoshuaGreen Dec 13, 2023
8f7a900
Things simplify if we assume that fxfMINSIZE > 0, so let's do that.
JoshuaGreen Dec 14, 2023
3eaa9d1
Adding the buffer option.
JoshuaGreen Dec 14, 2023
ab293f8
Choosing a more descriptive name.
JoshuaGreen Jan 14, 2024
8d84958
We operate on dhtKeys through their values, so make that clear.
JoshuaGreen Jan 14, 2024
bcc1958
Restoring an assert.
JoshuaGreen Jan 28, 2024
be528c5
Keeping the original format of the assert.
JoshuaGreen Jan 28, 2024
61f3233
Marking the new assert as a TODO.
JoshuaGreen Jan 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 50 additions & 38 deletions DHT/dht.c
Original file line number Diff line number Diff line change
Expand Up @@ -396,14 +396,14 @@ static InternHsElement *stepDirTable(dirEnumerate *enumeration)

typedef struct
{
dhtHashValue (*Hash)(dhtConstValue);
int (*Equal)(dhtConstValue, dhtConstValue);
dhtConstValue (*DupKey)(dhtConstValue);
dhtConstValue (*DupData)(dhtConstValue);
void (*FreeKey)(dhtValue);
dhtHashValue (*Hash)(dhtKey);
int (*Equal)(dhtKey, dhtKey);
int (*DupKeyValue)(dhtValue, dhtValue *);
int (*DupData)(dhtValue, dhtValue *);
void (*FreeKeyValue)(dhtValue);
void (*FreeData)(dhtValue);
void (*DumpData)(dhtConstValue,FILE *);
void (*DumpKey)(dhtConstValue,FILE *);
void (*DumpData)(dhtValue,FILE *);
void (*DumpKeyValue)(dhtValue,FILE *);
} Procedures;

typedef struct dht {
Expand Down Expand Up @@ -436,7 +436,7 @@ typedef struct dht {
#define ActualLoadFactor(h) (((h)->KeyCount*100)/(h)->DirTab.count)
#endif /*OVERFLOW_SAVE*/

unsigned long dhtKeyCount(dht *h)
unsigned long dhtKeyCount(dht const *h)
{
return h->KeyCount;
}
Expand Down Expand Up @@ -508,17 +508,17 @@ dht *dhtCreate(dhtValueType KeyType, dhtValuePolicy KeyPolicy,
ht->procs.Hash= dhtProcedures[KeyType]->Hash;
ht->procs.Equal= dhtProcedures[KeyType]->Equal;
ht->procs.DumpData= dhtProcedures[DtaType]->Dump;
ht->procs.DumpKey= dhtProcedures[KeyType]->Dump;
ht->procs.DumpKeyValue= dhtProcedures[KeyType]->Dump;

if (KeyPolicy==dhtNoCopy)
{
ht->procs.DupKey= dhtProcedures[dhtSimpleValue]->Dup;
ht->procs.FreeKey= dhtProcedures[dhtSimpleValue]->Free;
ht->procs.DupKeyValue= dhtProcedures[dhtSimpleValue]->Dup;
ht->procs.FreeKeyValue= dhtProcedures[dhtSimpleValue]->Free;
}
else if (KeyPolicy==dhtCopy)
{
ht->procs.DupKey= dhtProcedures[KeyType]->Dup;
ht->procs.FreeKey= dhtProcedures[KeyType]->Free;
ht->procs.DupKeyValue= dhtProcedures[KeyType]->Dup;
ht->procs.FreeKeyValue= dhtProcedures[KeyType]->Free;
}

if (DataPolicy==dhtNoCopy)
Expand Down Expand Up @@ -559,8 +559,8 @@ void dhtDestroy(HashTable *ht)
while (b)
{
InternHsElement *tmp= b;
(ht->procs.FreeKey)((dhtValue)b->HsEl.Key);
(ht->procs.FreeData)((dhtValue)b->HsEl.Data);
(ht->procs.FreeKeyValue)(b->HsEl.Key.value);
(ht->procs.FreeData)(b->HsEl.Data);
b= b->Next;
FreeInternHsElement(tmp);
}
Expand Down Expand Up @@ -599,7 +599,7 @@ void dhtDumpIndented(int ind, HashTable *ht, FILE *f)
while (b)
{
fprintf(f, "%*s ", ind, "");
(ht->procs.DumpKey)(b->HsEl.Key, f);
(ht->procs.DumpKeyValue)(b->HsEl.Key.value, f);
fputs("->", f);
(ht->procs.DumpData)(b->HsEl.Data, f);
b= b->Next;
Expand Down Expand Up @@ -796,14 +796,18 @@ LOCAL void ShrinkHashTable(HashTable *ht)
shrinkDirTable(&ht->DirTab);
}

LOCAL InternHsElement **LookupInternHsElement(HashTable *ht, dhtConstValue key)
LOCAL InternHsElement **LookupInternHsElement(HashTable *ht, dhtKey key)
{
uLong h;
InternHsElement **phe;

TraceFunctionEntry(__func__);
TraceFunctionParam("%p",(void *)ht);
TraceFunctionParam("%p",(void const *)key);
#if (defined(__cplusplus) && (__cplusplus >= 201103L)) || (defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L))
TraceFunctionParam("%jx",key.value.unsigned_integer);
#else
TraceFunctionParam("%lx",(unsigned long)key.value.unsigned_integer);
#endif
TraceFunctionParamListEnd();

h = DynamicHash(ht->p, ht->maxp, (ht->procs.Hash)(key));
Expand All @@ -827,14 +831,18 @@ LOCAL InternHsElement **LookupInternHsElement(HashTable *ht, dhtConstValue key)
return phe;
}

void dhtRemoveElement(HashTable *ht, dhtConstValue key)
void dhtRemoveElement(HashTable *ht, dhtKey key)
{
MYNAME(dhtRemoveElement)
InternHsElement **phe, *he;

TraceFunctionEntry(__func__);
TraceFunctionParam("%p",(void *)ht);
TraceFunctionParam("%p",(void const *)key);
#if (defined(__cplusplus) && (__cplusplus >= 201103L)) || (defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L))
TraceFunctionParam("%jx",key.value.unsigned_integer);
#else
TraceFunctionParam("%lx",(unsigned long)key.value.unsigned_integer);
#endif
TraceFunctionParamListEnd();

phe= LookupInternHsElement(ht, key);
Expand All @@ -849,8 +857,8 @@ void dhtRemoveElement(HashTable *ht, dhtConstValue key)
ht->NextStep= ht->NextStep->Next;

*phe= he->Next;
(ht->procs.FreeData)((dhtValue)he->HsEl.Data);
(ht->procs.FreeKey)((dhtValue)he->HsEl.Key);
(ht->procs.FreeData)(he->HsEl.Data);
(ht->procs.FreeKeyValue)(he->HsEl.Key.value);
FreeInternHsElement(he);
ht->KeyCount--;
if (ActualLoadFactor(ht) < ht->MinLoadFactor)
Expand All @@ -877,21 +885,26 @@ void dhtRemoveElement(HashTable *ht, dhtConstValue key)
TraceFunctionResultEnd();
}

dhtElement *dhtEnterElement(HashTable *ht, dhtConstValue key, dhtConstValue data)
dhtElement *dhtEnterElement(HashTable *ht, dhtKey key, dhtValue data)
{
InternHsElement **phe, *he;
dhtConstValue KeyV;
dhtConstValue DataV;
dhtKey KeyV;
dhtValue DataV;

TraceFunctionEntry(__func__);
TraceFunctionParam("%p",(void *)ht);
TraceFunctionParam("%p",(void const *)key);
TraceFunctionParam("%p",(void const *)data);
#if (defined(__cplusplus) && (__cplusplus >= 201103L)) || (defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L))
TraceFunctionParam("%jx",key.value.unsigned_integer);
TraceFunctionParam("%jx",data.unsigned_integer);
#else
TraceFunctionParam("%lx",(unsigned long)key.value.unsigned_integer);
TraceFunctionParam("%lx",(unsigned long)data.unsigned_integer);
#endif
TraceFunctionParamListEnd();

assert(key!=0);
KeyV = (ht->procs.DupKey)(key);
if (KeyV==0)
assert(key.value.object_pointer!=0); /* TODO: This assert assumes that object_pointer is the active member.
Is there a more generic test we could do? Do we need one? */
if ((ht->procs.DupKeyValue)(key.value, &KeyV.value))
{
TraceText("key duplication failed\n");
TraceFunctionExit(__func__);
Expand All @@ -900,10 +913,9 @@ dhtElement *dhtEnterElement(HashTable *ht, dhtConstValue key, dhtConstValue data
return dhtNilElement;
}

DataV = data==0 ? 0 : (ht->procs.DupData)(data);
if (data!=0 && DataV==0)
if ((ht->procs.DupData)(data, &DataV))
{
(ht->procs.FreeKey)((dhtValue)KeyV);
(ht->procs.FreeKeyValue)(KeyV.value);
TraceText("data duplication failed\n");
TraceFunctionExit(__func__);
TraceFunctionResult("%p",(void *)dhtNilElement);
Expand All @@ -923,8 +935,8 @@ dhtElement *dhtEnterElement(HashTable *ht, dhtConstValue key, dhtConstValue data
TraceEOL();
if (he==0)
{
(ht->procs.FreeKey)((dhtValue)KeyV);
(ht->procs.FreeData)((dhtValue)DataV);
(ht->procs.FreeKeyValue)(KeyV.value);
(ht->procs.FreeData)(DataV);
TraceText("allocation of new intern Hs element failed\n");
TraceFunctionExit(__func__);
TraceFunctionResult("%p",(void *)dhtNilElement);
Expand All @@ -941,9 +953,9 @@ dhtElement *dhtEnterElement(HashTable *ht, dhtConstValue key, dhtConstValue data
else
{
if (ht->DtaPolicy == dhtCopy)
(ht->procs.FreeData)((dhtValue)he->HsEl.Data);
(ht->procs.FreeData)(he->HsEl.Data);
if (ht->KeyPolicy == dhtCopy)
(ht->procs.FreeKey)((dhtValue)he->HsEl.Key);
(ht->procs.FreeKeyValue)(he->HsEl.Key.value);
}

he->HsEl.Key = KeyV;
Expand Down Expand Up @@ -975,7 +987,7 @@ dhtElement *dhtEnterElement(HashTable *ht, dhtConstValue key, dhtConstValue data
return &he->HsEl;
}

dhtElement *dhtLookupElement(HashTable *ht, dhtConstValue key)
dhtElement *dhtLookupElement(HashTable *ht, dhtKey key)
{
InternHsElement **phe;
dhtElement *result;
Expand Down
36 changes: 18 additions & 18 deletions DHT/dht.h
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
* Institut fuer Informatik, TU Muenchen, Germany
* [email protected]
* You may use this code as you wish, as long as this
* comment with the above copyright notice is keept intact
* comment with the above copyright notice is kept intact
* and in place.
*/
typedef enum {
Expand All @@ -15,28 +15,28 @@ typedef enum {

/* Now finally this is the HashElement */
typedef struct {
dhtConstValue Key;
dhtConstValue Data;
dhtKey Key;
dhtValue Data;
} dhtElement;
#define dhtNilElement ((dhtElement *)0)
#define dhtNilElement ((dhtElement *)0)

struct dht;
#define dhtNilHashTable ((struct dht *)0)
#define dhtNilHashTable ((struct dht *)0)

/* procedures */
struct dht *dhtCreate(dhtValueType KeyType, dhtValuePolicy KeyPolicy,
dhtValueType DtaType, dhtValuePolicy DataPolicy);
dhtElement *dhtEnterElement(struct dht *, dhtConstValue key, dhtConstValue data);
unsigned int dhtBucketStat (struct dht *, unsigned int *counter, unsigned int n);
void dhtDestroy (struct dht *);
void dhtDump (struct dht *, FILE *);
void dhtDumpIndented (int i, struct dht *, FILE *);
void dhtRemoveElement (struct dht *, dhtConstValue key);
dhtElement *dhtLookupElement (struct dht *, dhtConstValue key);
dhtElement *dhtGetFirstElement(struct dht *);
dhtElement *dhtGetNextElement (struct dht *);
unsigned long dhtKeyCount (struct dht *);
char const *dhtErrorMsg (void);
struct dht *dhtCreate(dhtValueType KeyType, dhtValuePolicy KeyPolicy,
dhtValueType DtaType, dhtValuePolicy DataPolicy);
dhtElement *dhtEnterElement(struct dht *, dhtKey key, dhtValue data);
unsigned int dhtBucketStat(struct dht *, unsigned int *counter, unsigned int n);
void dhtDestroy(struct dht *);
void dhtDump(struct dht *, FILE *);
void dhtDumpIndented(int ind, struct dht *, FILE *);
void dhtRemoveElement(struct dht *, dhtKey key);
dhtElement *dhtLookupElement(struct dht *, dhtKey key);
dhtElement *dhtGetFirstElement(struct dht *);
dhtElement *dhtGetNextElement(struct dht *);
unsigned long dhtKeyCount(struct dht const *);
char const *dhtErrorMsg(void);

extern char dhtError[];

Expand Down
Loading